A method, system, device and medium for transcoding storage of magnetic tape data

Abstract

This application provides a method, system, device, and medium for transcoding and storing magnetic tape data, relating to the field of computer data management technology. The method includes: running a dump program on an AS400 server to read magnetic tape backup data from a physical magnetic tape; splitting the magnetic tape backup data into multiple sub-data to be transmitted and allocating them to multiple parallel transmission queues; starting multiple transmission processes to read the sub-data from the transmission queues in parallel and sending them to a transcoding server cluster; the transcoding server cluster receiving the sub-data in parallel, converting its encoding format into a file format readable by the target operating system, generating transcoded files, and storing them on a shared disk; after all transcoded files corresponding to the same physical magnetic tape are generated, backing them up to a storage server and deleting the corresponding files on the shared disk, so that subsequent recovery and reading of the magnetic tape backup data does not require restarting the AS400 server and physical magnetic tape, thus eliminating dependence on the AS400 environment.

Description

Technical Field

[0001] This application relates to the field of computer data management technology, and in particular to a method, system, device and medium for transcoding and storing magnetic tape data. Background Technology

[0002] In the process of informatization in the financial industry, the IBM AS400 minicomputer was the mainstream operating system for a long time, generating a large amount of business data backed up on magnetic tapes during its decades of operation. With the development of information technology security and controllability, the industry has generally adopted open source and domestically developed products to replace the AS400 system. However, the data on these tapes faces difficulties in reading: there is currently a lack of open source or domestically developed solutions on the market that can directly read AS400 tape format. Data reading still heavily relies on the original AS400 operating system and dedicated tape drives, forming a continuous technical binding and compatibility barrier.

[0003] Existing Chinese patent CN114253474A uses the AS400 operating system's tape copy command to copy tape data to a virtual tape drive, and then sends the corresponding files on the virtual tape to a relay Linux host via FTP. The NBU backup system is used to back up the tape files, but the generated backup files still require the AS400 environment for parsing, thus not fundamentally eliminating dependence on the original system. Furthermore, using magnetic tape as a storage medium carries a long-term risk of demagnetization, potentially leading to permanent data loss. The AS400 minicomputer and related hardware are gradually being phased out of the mainstream market, making maintenance and hardware replacement increasingly difficult, further increasing the potential risks to long-term data storage and access.

[0004] Therefore, there is an urgent need for a technical solution that can directly transcode and reliably store magnetic tape data independently of the AS400 environment. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method, system, device, and medium for transcoding and storing magnetic tape data. The aim is to convert AS400 magnetic tape data into a file format directly readable on Linux / Windows operating systems and then back it up for storage. Ultimately, this allows the recovery and reading of magnetic tape data to be completely independent of the AS400 environment, achieving storage and access under a general operating system environment. The technical solution is as follows: Firstly, a method for transcoding and storing magnetic tape data is provided, including: Run a dump program on the AS400 server to read the tape backup data in the physical tape, split the tape backup data into multiple sub-data to be transmitted according to preset rules, and allocate them to multiple parallel transmission queues; start multiple parallel transmission processes, read the sub-data to be transmitted from the multiple transmission queues in parallel, and send them to the transcoding server cluster. The transcoding server cluster starts multiple transcoding processes to receive and process the sub-data to be transmitted in parallel, convert its encoding format into a file format readable by the target operating system, generate transcoded files, and store them on a shared disk. In response to confirmation that all transcoded files corresponding to the same physical tape have been generated, the transcoded files are backed up to the storage server using a backup tool, and the corresponding transcoded files on the shared disk are deleted after the backup is completed.

[0006] In one possible implementation, the AS400 server obtains the tape number of the physical tape, and when running the dump program, the dump program locates and reads the corresponding tape backup data based on the tape number.

[0007] In one possible implementation, the preset rule is to split the data according to the data block size or time slice.

[0008] In one possible implementation, the transcoding server cluster comprises multiple relay machines, each running multiple transcoding processes to process sub-data to be transmitted from different sending queues in parallel.

[0009] In one possible implementation, the target operating system is either Linux or Windows.

[0010] In one possible implementation, the method further includes: when it is necessary to access tape backup data on a physical tape, restoring the transcoded files stored on a storage server to a Linux or Windows operating system using the backup tool.

[0011] Secondly, a magnetic tape data transcoding and storage system is provided, the system being used to execute the magnetic tape data transcoding and storage method described in any of the preceding claims, including: The AS400 server includes a tape data recovery module, a tape data splitting module, a transmission queue, and a transmission driver module. A dump program is deployed in the AS400 server. This program reads the tape backup data from the physical tape and restores it to the tape data recovery module. The tape data recovery module inputs the restored tape backup data into the tape data splitting module. The tape data splitting module splits the tape backup data into multiple sub-data to be transmitted according to preset rules and allocates them to multiple parallel transmission queues. Multiple parallel transmission processes in the transmission driver module are started to read the sub-data to be transmitted from the multiple transmission queues in parallel and send them to the transcoding server cluster. The transcoding server cluster includes multiple relay machines, each running a transcoding process and a backup process. The transcoding process receives and processes the sub-data to be transmitted, converts its encoding format to a file format readable by the target operating system, generates transcoded files, and stores them on a shared disk. The backup process, once it is confirmed that all transcoded files corresponding to the same physical tape have been generated, backs up the transcoded files to a storage server using a backup tool, and deletes the corresponding transcoded files from the shared disk after the backup is complete.

[0012] Thirdly, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform a magnetic tape data transcoding and storage method as described in any of the preceding claims.

[0013] Fourthly, a storage medium is provided that stores a computer program, wherein the computer program is configured to execute, at runtime, a magnetic tape data transcoding and storage method as described in any of the preceding claims.

[0014] The technical solutions provided in this application can achieve the following technical effects: (1) First, the dump program is run on the AS400 server to read and split the tape backup data in the physical tape. Then, the transcoding server cluster converts it into a standard file that can be read by Linux / Windows. Finally, the tape backup data is completely separated from the original proprietary encoding format, so as to fundamentally achieve the decoupling of the data carrier and the operating environment. This makes it possible to restore and read the tape backup data without restarting the AS400 server and physical tape. This effectively solves the risk of data unreadable caused by the obsolescence of the original AS400 minicomputer system hardware and the interruption of maintenance supply.

[0015] (2) Real-time detection of the transcoding task completion status of the transcoding server cluster, and after confirming that all tape backup data belonging to the same physical disk has been transcoded, the backup tool is automatically triggered to archive the transcoded files to the storage server, and then the corresponding temporary files on the shared disk are deleted. This not only achieves permanent and reliable storage of the transcoded files, but also ensures that the system storage resources can be used cyclically and efficiently for subsequent batch tape backup data transcoding tasks because the storage space of the shared disk is automatically released.

[0016] (3) By adopting a processing architecture of multiple parallel sending queues, multiple transcoding process collaboration and clustered deployment of relay machines, it supports splitting the tape backup data of a single physical tape and transmitting and transcoding it concurrently, or can process the tape backup data in multiple physical tapes at the same time, so as to improve the throughput and processing efficiency of tape backup data transcoding and storage, and meet the practical needs of migrating and saving massive historical tape backup data within a limited time. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. In the drawings: Figure 1 This is a block diagram of a magnetic tape data transcoding and storage system provided in an embodiment of this application; Figure 2 This is a flowchart of a method for transcoding and storing magnetic tape data provided in an embodiment of this application; Figure 3 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0020] To enable direct transcoding and long-term reliable storage of magnetic tape data independent of the AS400 environment, this application provides a magnetic tape data transcoding and storage system, such as... Figure 1 As shown, the system includes an AS400 server, a transcoding server cluster, a shared disk, and a storage server.

[0021] The AS400 server is the source host that performs tape data reading and preliminary processing. Physically connected to this server are tape drives, which are dedicated external storage devices for reading and writing physical tapes, which are the magnetic media storing historical tape backup data. During operation, the operator inserts the physical tape to be converted into the tape drive, and the AS400 server controls and accesses the tape drive through its dedicated hardware interface and automated program, thereby reading the tape backup data from the physical tape.

[0022] The AS400 server internally comprises a recovery tape data module, a split tape data module, a send queue, and a send driver module, all connected in series. The recovery tape data module primarily receives and temporarily stores the raw tape backup data read from the physical tapes. To automatically read tape backup data from the physical tapes, the AS400 server deploys a dump program. This dump program can be a script with dump functionality, including web applications and SQL scripts. The dump program is configured to, when run, receive the unique tape number of the physical tape from operator input or a configuration file. Based on this tape number, the dump program drives the tape drive to locate the corresponding physical tape and automatically reads all the tape backup data from that physical tape. It then restores the read tape backup data and transmits it to the local recovery tape data module on the AS400 server.

[0023] In a specific example, the AS400 server can periodically run a dump program, traversing a pre-defined list of tape numbers, to perform batch, periodic read operations on tape backup data across multiple physical tapes.

[0024] In another specific example, when an operator connects a new physical tape to a tape drive connected to the AS400 server, the dump program can be manually triggered or automatically detected and triggered by the system. The dump program can automatically identify the tape number or prompt the operator to enter the tape number, and then read the tape backup data from the connected physical tape.

[0025] After receiving the complete tape backup data, the tape recovery module transmits it to the tape splitting module. The tape splitting module splits the complete backup data into multiple sub-data blocks to be transmitted according to preset rules (such as by data block size or time slice), and distributes these sub-data blocks to multiple parallel transmission queues.

[0026] The send driver module monitors the status of each send queue and starts multiple parallel send processes. Each send process reads the sub-data to be transmitted from its bound send queue and sends it to the transcoding server cluster over the network.

[0027] A transcoding server cluster consists of multiple relay servers, such as Figure 1 The system consists of n relay servers, where n ≥ 2. Each relay server runs a transcoding process and a backup process. After receiving sub-data from the AS400 server, the transcoding server cluster starts transcoding processes on multiple relay servers, i.e., it starts multiple parallel transcoding processes. Each transcoding process is responsible for converting the received sub-data from its original AS400 proprietary encoding format to a file format readable by the target operating system, generating a transcoded file, and writing it to a specified directory on a shared disk. It should be noted that the files generated from the transcoding of all sub-data on the same physical tape are logically stored together.

[0028] The shared disk is a network storage device accessible to both the AS400 server and the transcoding server cluster. It serves as a temporary storage device for transcoded files, acting as a data buffer. The transcoding server cluster has a monitoring program deployed within it. This program, a script with monitoring capabilities, periodically scans the shared disk to check if all transcoded files corresponding to each physical tape have been generated. Once the transcoding of the tape backup data for the same physical tape is confirmed to be complete, the monitoring program triggers a backup process. This process calls a backup tool to package and back up all transcoded files corresponding to that physical tape on the shared disk to a remote storage server for long-term archiving. After successful backup verification, the shared disk automatically deletes the corresponding temporary files to free up storage space.

[0029] Therefore, it can be seen that after the system of this application processes the tape backup data in the physical tape according to the above process, the tape backup data is securely stored in the storage server in a file format readable by the target operating system. Subsequently, if there is a need to query or restore the tape backup data, the required files can be directly read by restoring them to any target operating system environment using a backup tool, without needing to restart the original AS400 server and tape drive. This completely eliminates dependence on the AS400 environment and ensures long-term readability and reliable data storage.

[0030] It should be noted that the target operating system in this embodiment is a general Linux or Windows operating system.

[0031] To more clearly illustrate the transcoding process of tape backup data in the tape data transcoding and storage system of this application, this application also provides a tape data transcoding and storage method, which is applied to the aforementioned tape data transcoding and storage system. For example... Figure 2 As shown, a method for transcoding and storing magnetic tape data mainly includes steps S101 to S104.

[0032] Step S101: Run a dump program on the AS400 server to read the tape backup data in the physical tape, split the tape backup data into multiple sub-data to be transmitted according to preset rules, and allocate them to multiple parallel transmission queues.

[0033] A dump program pre-deployed on the AS400 server receives the input tape number, drives the tape drive to locate and read the tape backup data from the corresponding physical tape, and restores it to the local storage of the AS400 server, specifically to the recovery tape data module. Subsequently, according to a preset splitting rule, the read tape backup data is divided into multiple independent sub-data to be transmitted. These sub-data are evenly distributed into multiple parallel transmission queues, preparing for subsequent high-concurrency transmissions.

[0034] Specifically, the splitting rules can be based on data blocks of fixed size, or on time slices divided by timestamps, where tape backup data received within a certain period is treated as a single sub-data segment to be transmitted. Of course, other splitting rules can also be defined, such as splitting based on data type or data source.

[0035] Step S102: Start multiple parallel sending processes, read the sub-data to be transmitted from multiple sending queues in parallel, and send it to the transcoding server cluster.

[0036] The sending queue is a logical cache structure created on the AS400 server to temporarily store sub-data to be transmitted and manage it according to the first-in, first-out principle. This embodiment sets up multiple sending queues, each corresponding to a parallel sending process. Each sending queue operates independently to ensure that the sub-data to be transmitted can be evenly distributed and achieve conflict-free parallel reading and transmission.

[0037] After the sending driver module detects data in the sending queue, it dynamically starts multiple independent sending processes. Each sending process is bound to a specific sending queue and continuously reads the sub-data to be transmitted from its bound queue. Multiple sending processes work in parallel, streaming the read sub-data to the transcoding server cluster over the network. In a specific example, the transmission process uses the FTP protocol to send the sub-data to be transmitted from the sending queue to the receiving directory in the transcoding server cluster, and generates a timestamped marker file upon completion of the transmission to mark the completion of a single transmission task. The parallel sending mechanism used in this embodiment fully utilizes network bandwidth, significantly improving the throughput efficiency of exporting sub-data from the AS400 system.

[0038] In step S103, the transcoding server cluster starts multiple transcoding processes to receive and process the sub-data to be transmitted in parallel, convert its encoding format into a file format readable by the target operating system, generate transcoded files, and store them on a shared disk.

[0039] After receiving the transmitted sub-data, the transcoding server cluster starts multiple transcoding processes on each relay machine. Each transcoding process works independently, responsible for receiving and processing one sub-data stream. The core task of the transcoding process is to decode and re-encode the sub-data from its original proprietary encoding format, which depends on the AS400 system, into a standard file format that the target operating system can directly recognize—that is, transcoding it into a standard file format that Linux or Windows operating systems can directly recognize. After transcoding is complete, each sub-data is generated as an independent transcoded file and written to a designated directory on a shared disk accessible to all relay machines. Multiple files generated from the same physical tape remain logically related.

[0040] In step S104, in response to confirming that all transcoded files corresponding to the same physical tape have been generated, the transcoded files are backed up to the storage server using a backup tool, and the corresponding transcoded files on the shared disk are deleted after the backup is completed.

[0041] A monitoring program deployed in the transcoding server cluster continuously scans the shared disks, checking file naming rules, metadata, or completion markers to determine whether all transcoded files associated with the same physical tape have been generated. Once the transcoding of the same physical tape is confirmed to be complete, the monitoring program automatically triggers a backup process. This process uses integrated backup tools to package all transcoded files of that batch on the shared disk and back them up completely to a remote dedicated storage server for long-term, secure archiving. After successful backup verification, the shared disk automatically deletes the corresponding temporary files, promptly releasing disk space and ensuring the reusable transfer buffer.

[0042] It should be noted that steps S101 to S104 in this embodiment are automatically linked and executed in parallel. They are primarily used to convert the unreadable AS400 tape backup data into standard files within the target operating system environment, enabling long-term archiving. In other words, after completing the dumping of tape backup data for all physical tapes, subsequent access, querying, or recovery of the tape backup data can be directly achieved within a Linux or Windows environment using backup tools. This completely eliminates dependence on the original AS400 hardware and operating system environment, fundamentally solving the risks and storage problems associated with reading and preserving tape backup data due to outdated technology.

[0043] It should also be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In practical applications, all the above possible implementation methods can be arbitrarily combined in a combined manner to form possible embodiments of this application, which will not be described in detail here.

[0044] Based on the same inventive concept, this application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute a magnetic tape data transcoding and storage method of any of the above embodiments.

[0045] In an exemplary embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 300 includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device 300 does not constitute a limitation on the embodiments of this application.

[0046] Processor 301 may be a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0047] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0048] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0049] The memory 303 stores computer program code that executes the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the computer program code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0050] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0051] Based on the same inventive concept, this application also provides a storage medium storing a computer program, wherein the computer program is configured to execute a magnetic tape data transcoding and storage method of any of the above embodiments when running.

[0052] Those skilled in the art will clearly understand that the specific working process of the systems, devices, and modules described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0053] Those skilled in the art will understand that the technical solution of this application, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several program instructions to cause an electronic device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of this application when running the program instructions. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0054] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as electronic devices like personal computers, servers, or network devices) associated with program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of this application.

[0055] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of this application, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to leave the protection scope of this application.

Claims

1. A method for transcoding and storing magnetic tape data, characterized in that, include: Run a dump program on the AS400 server to read the tape backup data in the physical tape, split the tape backup data into multiple sub-data to be transmitted according to preset rules, and allocate them to multiple parallel transmission queues. Multiple parallel sending processes are initiated to read the sub-data to be transmitted from the multiple sending queues in parallel and send it to the transcoding server cluster; The transcoding server cluster starts multiple transcoding processes to receive and process the sub-data to be transmitted in parallel, convert its encoding format into a file format readable by the target operating system, generate transcoded files, and store them on a shared disk. In response to confirmation that all transcoded files corresponding to the same physical tape have been generated, the transcoded files are backed up to the storage server using a backup tool, and the corresponding transcoded files on the shared disk are deleted after the backup is completed.

2. The method according to claim 1, characterized in that, The AS400 server obtains the tape number of the physical tape, and when running the dump program, the dump program locates and reads the corresponding tape backup data based on the tape number.

3. The method according to claim 1 or 2, characterized in that, The preset rule is to split the data according to the data block size or time slice.

4. The method according to claim 1, characterized in that, The transcoding server cluster contains multiple relay machines, each running multiple transcoding processes to process sub-data to be transmitted from different sending queues in parallel.

5. The method according to claim 1, characterized in that, The target operating system is either Linux or Windows.

6. The method according to claim 5, characterized in that, The method further includes: when it is necessary to access the tape backup data in the physical tape, restoring the transcoded files stored in the storage server to the Linux or Windows operating system using the backup tool.

7. A magnetic tape data transcoding and storage system, used to perform the magnetic tape data transcoding and storage method as described in any one of claims 1 to 6, characterized in that, include: The AS400 server includes a tape data recovery module, a tape data splitting module, a send queue, and a send driver module; The AS400 server is equipped with a dump program that reads the tape backup data from the physical tape and restores the tape backup data to the recovery tape data module. The recovery tape data module inputs the restored tape backup data into the split tape data module, which then splits the tape backup data into multiple sub-data to be transmitted according to preset rules and allocates them to multiple parallel transmission queues. Multiple parallel sending processes are initiated in the sending driver module to read the sub-data to be transmitted from the multiple sending queues in parallel and send it to the transcoding server cluster; The transcoding server cluster includes multiple relay machines, each running a transcoding process and a backup process. The transcoding process receives and processes the sub-data to be transmitted, converts its encoding format to a file format readable by the target operating system, generates transcoded files, and stores them on a shared disk. The backup process, once it is confirmed that all transcoded files corresponding to the same physical tape have been generated, backs up the transcoded files to a storage server using a backup tool, and deletes the corresponding transcoded files from the shared disk after the backup is complete.

8. An electronic device, characterized in that, The device includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the transcoding and storage method for magnetic tape data according to any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the transcoding and storage method for magnetic tape data according to any one of claims 1 to 6 when it is run.

Images