File preservation method, device and system based on distributed platform

Abstract

The application provides a file preservation method, device and system based on a distributed platform, relates to the technical field of cloud computing, and can be applied to the technical field of finance or other technical fields.The file preservation method based on the distributed platform comprises the following steps: acquiring a platform file, a host file and a running stage;generating a check file according to the platform file, the host file and the running stage;selecting a corresponding preservation script according to the running stage;preserving the check file by using the preservation script, and generating a preservation file;and pushing the preservation file to a downstream application.The application can effectively reduce the workload of downstream replacement and the error probability of large-scale replacement logic, and reduce the occurrence of production accidents.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and more specifically, to a file anti-corrosion method, apparatus, and system based on a distributed platform. Background Technology

[0002] Technical solutions of existing technologies

[0003] With the promotion of distributed platform technology in banks, data that was originally provided solely by the mainframe is now provided by various distributed platforms. The data provider has changed from one to multiple or even dozens of providers.

[0004] Disadvantages of existing technology

[0005] First, there are significant time differences between platforms; second, the data is too scattered, resulting in very poor usability. The presence of multiple data providers forces users to undertake a great deal of adjustment and switching work, and even causes logical errors and large-scale production accidents due to too many data sources. Summary of the Invention

[0006] The main objective of this invention is to provide a file anti-corrosion method, device, and system based on a distributed platform, so as to shield the timeliness and structural differences in data exchange between multiple platforms, effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, and reduce the occurrence of production accidents.

[0007] To achieve the above objectives, embodiments of the present invention provide a file corruption prevention method based on a distributed platform, comprising:

[0008] Obtain platform files, host files, and runtime phases;

[0009] A verification file is generated based on the platform file, the host file, and the runtime phase;

[0010] Based on the aforementioned operational stage, select the corresponding anti-corrosion script to perform anti-corrosion on the verification file and generate an anti-corrosion file;

[0011] The anti-corrosion document is pushed to downstream applications.

[0012] In one embodiment, generating a checklist based on the platform file, the host file, and the runtime phase includes:

[0013] Compare the platform file and the host file to generate a comparison result;

[0014] The verification file is generated based on the comparison results and the running phase.

[0015] In one embodiment, the verification files include a platform verification file and a host verification file;

[0016] The verification file is generated based on the comparison results and the operational phase, including:

[0017] The number of platform verification files and the number of host verification files are determined based on the comparison results and the operational phase.

[0018] The platform verification file is determined based on the number of platform files and platform verification files, and the host verification file is determined based on the number of host files and host verification files.

[0019] In one embodiment, comparing the platform file and the host file to generate a comparison result includes:

[0020] The comparison method is determined based on the structure of the platform file and the structure of the host file;

[0021] The platform file and the host file are compared according to the comparison method to generate the comparison result.

[0022] In one embodiment, it further includes:

[0023] Receive customer business instructions and insert corresponding records into the log according to the transaction type in the customer business instructions;

[0024] Verify the customer's business instructions, and generate the platform file based on the verification results and the records.

[0025] This invention also provides a file anti-corrosion device based on a distributed platform, comprising:

[0026] The acquisition module is used to acquire platform files, host files, and runtime phases.

[0027] The verification file module is used to generate a verification file based on the platform file, the host file, and the runtime phase.

[0028] The anti-corrosion module is used to select the corresponding anti-corrosion script according to the running stage to perform anti-corrosion on the verification file and generate an anti-corrosion file;

[0029] The file push module is used to push the anti-corrosion file to downstream applications.

[0030] In one embodiment, the document verification module includes:

[0031] The comparison unit is used to compare the platform file and the host file and generate a comparison result;

[0032] The verification file unit is used to generate the verification file based on the comparison results and the running stage.

[0033] In one embodiment, the verification files include a platform verification file and a host verification file;

[0034] The document verification unit includes:

[0035] The file quantity verification subunit is used to determine the number of platform-verified files and the number of host-verified files based on the comparison results and the running stage.

[0036] The verification file subunit is used to determine the platform verification file based on the platform file and the number of platform verification files, and to determine the host verification file based on the host file and the number of host verification files.

[0037] In one embodiment, the comparison unit includes:

[0038] The comparison method subunit is used to determine the comparison method based on the structure of the platform file and the structure of the host file;

[0039] The comparison result subunit is used to compare the platform file and the host file according to the comparison method and generate the comparison result.

[0040] In one embodiment, it further includes:

[0041] The record insertion module is used to receive customer business instructions and insert corresponding records into the log according to the transaction type in the customer business instructions.

[0042] The platform file generation module is used to verify the customer business instructions and generate the platform file based on the verification results and the records.

[0043] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the file anti-corrosion method based on a distributed platform.

[0044] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the file anti-corrosion method based on a distributed platform.

[0045] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the file anti-corrosion method based on a distributed platform.

[0046] This invention also provides a file corruption prevention system based on a distributed platform, comprising:

[0047] The file preservation device based on the distributed platform described above is applied to the platform;

[0048] The host is used to send host files to the file preservation device;

[0049] Downstream applications are used to receive the anti-corrosion documents pushed by the document anti-corrosion device.

[0050] The file anti-corrosion method, apparatus and system based on a distributed platform in this invention first generates a verification file based on platform files, host files and runtime stages, then selects the corresponding anti-corrosion script to perform anti-corrosion on the verification file according to the runtime stage, and finally pushes the generated anti-corrosion file to downstream applications. This can effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, and reduce the occurrence of production accidents. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of a file anti-corrosion method based on a distributed platform in an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the anti-corrosion treatment logic in an embodiment of the present invention;

[0054] Figure 3 This is a flowchart illustrating the generation of platform files in an embodiment of the present invention;

[0055] Figure 4 This is a flowchart of S102 in an embodiment of the present invention;

[0056] Figure 5 This is a flowchart of S201 in an embodiment of the present invention;

[0057] Figure 6 This is a flowchart of S202 in an embodiment of the present invention;

[0058] Figure 7 This is a structural block diagram of the file anti-corrosion device based on a distributed platform in an embodiment of the present invention;

[0059] Figure 8 This is a structural block diagram of the data integration module in an embodiment of the present invention;

[0060] Figure 9 This is a structural block diagram of the open platform server cluster in an embodiment of the present invention;

[0061] Figure 10 This is a structural block diagram of the data lake in an embodiment of the present invention;

[0062] Figure 11 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application.

[0063] Figure 12 This is a file anti-corrosion system based on a distributed platform in this embodiment of the invention;

[0064] Figure 13 This is a structural block diagram of the host in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0067] The acquisition, storage, use, and processing of data in the technical solution of this invention all comply with relevant national laws and regulations. The user information in the embodiments of this application was obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have all been authorized and agreed upon by the client.

[0068] This invention shields the timeliness and structural differences in data exchange across multiple platforms, thereby improving the problem of frequent modifications by downstream data users. Based on Hadoop, this invention transforms the original many-to-many data distribution to multiple downstream users into a many-to-one and then many-to-many distribution. First, multiple data sources are pasted into Hadoop, and then, through certain logic (closed within each data source), an anti-corrosion layer is established after each of the multiple data sources. The data in this anti-corrosion layer is identical to the original single-source data. This anti-corrosion layer achieves synchronization from the data source to the layer in real-time and near real-time modes, and ensures that the timeliness and structure of the anti-corrosion layer remain consistent with the original single data source in terms of dimensional design, record uniqueness, attribute design, and extended attribute design. Simultaneously, by utilizing extended record time fields, the anti-corrosion layer distributes files in the same way as the original single-source data source. The logic for file output and file content remain unchanged at the application end, shielding downstream data users from modifications; only file replacement is required, improving the efficiency of file users and supporting flexible adjustments according to regulatory needs.

[0069] Figure 1 This is a flowchart of a file anti-corrosion method based on a distributed platform in an embodiment of the present invention. Figure 2 This is a schematic diagram of the anti-corrosion treatment logic in an embodiment of the present invention. For example... Figures 1-2 As shown, file corruption prevention methods based on a distributed platform include:

[0070] S101: Obtain platform files, host files, and runtime phases.

[0071] Figure 3 This is a flowchart illustrating the generation of platform files in an embodiment of the present invention. For example... Figure 3 As shown, file corruption prevention methods based on distributed platforms also include:

[0072] S201: Receive customer business instructions and insert corresponding records into the log according to the transaction type in the customer business instructions.

[0073] S202: Verify the customer business instruction, and generate the platform file based on the verification result and the record.

[0074] In practice, the transaction elements in the platform files are based on the basic data and anti-corrosion elements required by downstream banking reports / details and regulatory reporting. When the verification result is correct, the record is updated, and the record status is changed to intermediate state. When the business is completed or closed, the record is updated again, and the record status is changed to final state. The platform files are generated based on the updated records.

[0075] S102: Generate a verification file based on the platform file, the host file, and the runtime phase.

[0076] Figure 4 This is a flowchart of S102 in an embodiment of the present invention. For example... Figure 4 As shown, S102 includes:

[0077] S301: Compare the platform file and the host file to generate a comparison result.

[0078] Figure 5 This is a flowchart of S201 in an embodiment of the present invention. For example... Figure 5 As shown, S201 includes:

[0079] S401: Determine the comparison method based on the structure of the platform file and the structure of the host file.

[0080] S402: Compare the platform file and the host file according to the comparison method, and generate the comparison result.

[0081] In practice, based on the downstream usage logic and file operation principles of lake surface data, the focus is on comparing the basic elements of banking business transactions, including amount, time, transaction information and location, and comparing the status of records one by one.

[0082] When the host lake table (host file) and the platform lake table (platform file) have the same structure, a full table field-by-field comparison is used to compare the platform file and the host file. When the host lake table (host file) and the platform lake table (platform file) have different structures, a full comparison at the same dimension is used to compare the platform file and the host file.

[0083] S302: Generate the verification file based on the comparison results and the running phase.

[0084] In practical implementation, within the bank's core system, ensuring the continuous and stable operation of business is the priority. During the process of removing and replacing the host, the principle of consistency is used as the underlying logic of the anti-corrosion technology. File consistency comparison from a single data source to multiple data sources is achieved in three stages: the dual-machine parallel stage based on the host, the dual-machine parallel stage based on the platform, and the dual-write stage. Specifically, this involves comparing files on the host platform. Based on the consistency comparison results, corresponding anti-corrosion scripts are loaded to achieve file consistency across multiple data sources, ensuring the continuous and stable operation of the commercial bank's core business system from end to end.

[0085] The verification documents include platform verification documents and host verification documents.

[0086] Figure 6 This is a flowchart of S202 in an embodiment of the present invention. For example... Figure 6 As shown, S202 includes:

[0087] S501: Determine the number of platform verification files and the number of host verification files based on the comparison results and the running phase.

[0088] In practice, when the operation phase is host-based, most comparison results are based on the host, while a small portion is based on the platform, with fewer platform-verified files than host-verified files. When the operation phase is dual-machine parallel, most comparison results are based on the platform, while a small portion is based on the host, with more platform-verified files than host-verified files. When the operation phase is dual-write, the platform is the standard, and all comparison results are platform-verified files.

[0089] S502: Determine the platform verification file based on the number of platform files and platform verification files, and determine the host verification file based on the number of host files and host verification files.

[0090] S103: Select the corresponding anti-corrosion script according to the running stage to perform anti-corrosion on the verification file and generate an anti-corrosion file.

[0091] In specific implementation, the present invention adds an anti-corrosion layer to the lake surface with the goal of stable operation during the host process. Different anti-corrosion processing scripts are made for the three different stages (host-based stage, dual-machine parallel platform-based stage, and disconnected dual-write stage). Based on the results of multi-source data verification, with consistency as the underlying logic, corresponding anti-corrosion processing is adopted to ensure that the downstream processing of multiple data sources is consistent.

[0092] S104: Push the anti-corrosion file to downstream applications.

[0093] In practice, anti-corrosion documents can be generated in single or multiple sessions and pushed downstream in real-time or near real-time. The anti-corrosion documents are the final data entering the data lake, and can be batch-processed according to customer details (real-time, historical) and regulatory reporting (full, incremental) to support downstream display or reporting.

[0094] Figure 1 The document preservation method based on a distributed platform shown can be implemented by a document preservation device applied to a distributed platform. Figure 1 As shown in the process, the file anti-corrosion method based on a distributed platform in this embodiment of the invention first generates a verification file based on the platform file, host file, and runtime stage. Then, it selects the corresponding anti-corrosion script based on the runtime stage to perform anti-corrosion on the verification file. Finally, it pushes the generated anti-corrosion file to downstream applications. This can effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, thereby reducing the occurrence of production accidents.

[0095] Based on the same inventive concept, this invention also provides a file anti-corrosion device based on a distributed platform. Since the principle of this device in solving the problem is similar to that of the file anti-corrosion method based on a distributed platform, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be described again.

[0096] Figure 7 This is a structural block diagram of a file anti-corrosion device based on a distributed platform, as described in an embodiment of the present invention. Figure 7 As shown, the file corruption prevention device based on a distributed platform includes:

[0097] The acquisition module is used to acquire platform files, host files, and runtime phases.

[0098] The verification file module is used to generate a verification file based on the platform file, the host file, and the runtime phase.

[0099] The anti-corrosion module is used to select the corresponding anti-corrosion script according to the running stage to perform anti-corrosion on the verification file and generate an anti-corrosion file;

[0100] The file push module is used to push the anti-corrosion file to downstream applications.

[0101] In one embodiment, the document verification module includes:

[0102] The comparison unit is used to compare the platform file and the host file and generate a comparison result;

[0103] The verification file unit is used to generate the verification file based on the comparison results and the running stage.

[0104] In one embodiment, the verification files include a platform verification file and a host verification file;

[0105] The document verification unit includes:

[0106] The file quantity verification subunit is used to determine the number of platform-verified files and the number of host-verified files based on the comparison results and the running stage.

[0107] The verification file subunit is used to determine the platform verification file based on the platform file and the number of platform verification files, and to determine the host verification file based on the host file and the number of host verification files.

[0108] In one embodiment, the comparison unit includes:

[0109] The comparison method subunit is used to determine the comparison method based on the structure of the platform file and the structure of the host file;

[0110] The comparison result subunit is used to compare the platform file and the host file according to the comparison method and generate the comparison result.

[0111] In one embodiment, it further includes:

[0112] The record insertion module is used to receive customer business instructions and insert corresponding records into the log according to the transaction type in the customer business instructions.

[0113] The platform file generation module is used to verify the customer business instructions and generate the platform file based on the verification results and the records.

[0114] In practical applications, file corruption prevention devices based on distributed platforms include data integration modules, open platform server clusters, and data lakes. Figure 8 This is a structural block diagram of the data integration module in an embodiment of the present invention. Figure 9 This is a structural block diagram of the open platform server cluster in an embodiment of the present invention. Figure 10 This is a structural block diagram of the data lake in an embodiment of the present invention. For example... Figures 8-10 As shown:

[0115] Data Integration Module 1 includes an acquisition module, a document verification module, and an anti-corrosion module. It is responsible for receiving business transaction data files from various channels within the bank and performing anti-corrosion processing on the files according to required processing needs. This module is further divided into a data generation module 6, a data acquisition module 7, and a data processing judgment module 8. Data Integration Module 1 can determine whether the acquired data requires anti-corrosion processing based on the current data structure logic. If anti-corrosion processing is required, Data Integration Module 1 will switch the data acquired from the current business transaction to the data lake anti-corrosion processing center for centralized processing.

[0116] The open platform server cluster 2 includes a file push module, a record insertion module, and a platform file generation module. It contains platform table data routing content, covering data routed through the platform, business processing logic, cache storage, and table storage. This establishes a basic support platform and data storage that supports closed-loop operation of business within the development platform, providing data support for various anti-corrosion processing needs. The data routing module 21 defines basic data such as corresponding account cards through platform routing, enabling transactions to run in a closed loop within the open platform. The business processing module 22 initiates specific online or batch business processing, updates the cache module 23, and stores it in real-time in the data storage module 24. This module performs online or batch updates to the anti-corrosion preprocessing.

[0117] Data Lake 4 can collect data and copy data from business source systems into the lake for processing. Platform batch file collection module 41 performs platform batch file processing on data provided by platform data storage module 24. Host batch file collection module 42 performs host batch file processing on data provided by host batch processing module 34 and stores and processes the collected data. Storage integration and processing module 43 performs real-time processing or batch processing according to specific business and timeliness requirements.

[0118] In summary, the file anti-corrosion device based on a distributed platform in this embodiment of the invention first generates a verification file based on the platform file, host file, and runtime stage. Then, it selects the corresponding anti-corrosion script according to the runtime stage to perform anti-corrosion on the verification file. Finally, it pushes the generated anti-corrosion file to downstream applications. This can effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, thereby reducing the occurrence of production accidents.

[0119] Figure 11 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 11 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 11 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0120] In one embodiment, the file corruption prevention method based on a distributed platform can be integrated into the central processing unit 9100.

[0121] The central processing unit 9100 can be configured to perform the following controls:

[0122] Obtain platform files, host files, and runtime phases;

[0123] A verification file is generated based on the platform file, the host file, and the runtime phase;

[0124] Based on the aforementioned operational stage, select the corresponding anti-corrosion script to perform anti-corrosion on the verification file and generate an anti-corrosion file;

[0125] The anti-corrosion document is pushed to downstream applications.

[0126] As can be seen from the above description, the file anti-corrosion method based on a distributed platform provided in this application first generates a verification file based on the platform file, host file, and runtime stage. Then, it selects the corresponding anti-corrosion script to perform anti-corrosion on the verification file according to the runtime stage. Finally, it pushes the generated anti-corrosion file to downstream applications, which can effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, and reduce the occurrence of production accidents.

[0127] In another embodiment, the file preservation device based on the distributed platform can be configured separately from the central processing unit 9100. For example, the file preservation device based on the distributed platform can be configured as a chip connected to the central processing unit 9100, and the function of the file preservation method based on the distributed platform can be realized through the control of the central processing unit.

[0128] like Figure 11 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 11 All components shown; in addition, the electronic device 9600 may also include Figure 11 For components not shown, please refer to existing technologies.

[0129] like Figure 11 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0130] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0131] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0132] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer 9141 (sometimes referred to as a buffer memory). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0133] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0134] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via the antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0135] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0136] This invention also provides a computer-readable storage medium capable of implementing all steps of the file preservation method based on a distributed platform, where the execution subject is a server or client, as described in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the file preservation method based on a distributed platform as described in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0137] Obtain platform files, host files, and runtime phases;

[0138] A verification file is generated based on the platform file, the host file, and the runtime phase;

[0139] Based on the aforementioned operational stage, select the corresponding anti-corrosion script to perform anti-corrosion on the verification file and generate an anti-corrosion file;

[0140] The anti-corrosion document is pushed to downstream applications.

[0141] In summary, the computer-readable storage medium of this invention first generates a verification file based on the platform file, host file, and runtime phase. Then, it selects the corresponding anti-corrosion script according to the runtime phase to perform anti-corrosion on the verification file. Finally, it pushes the generated anti-corrosion file to downstream applications. This can effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, thereby reducing the occurrence of production accidents.

[0142] This invention also provides a computer program product capable of implementing all steps of the file corruption prevention method based on a distributed platform, where the execution subject is a server or client, as described in the above embodiments. The computer program product includes a computer program / instruction, which, when executed by a processor, implements all steps of the file corruption prevention method based on a distributed platform as described in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0143] Obtain platform files, host files, and runtime phases;

[0144] A verification file is generated based on the platform file, the host file, and the runtime phase;

[0145] Based on the aforementioned operational stage, select the corresponding anti-corrosion script to perform anti-corrosion on the verification file and generate an anti-corrosion file;

[0146] The anti-corrosion document is pushed to downstream applications.

[0147] In summary, the computer program product of this invention first generates a verification file based on the platform file, host file, and runtime phase. Then, it selects the corresponding anti-corrosion script according to the runtime phase to perform anti-corrosion on the verification file. Finally, it pushes the generated anti-corrosion file to downstream applications. This can effectively reduce the workload of downstream replacement and the probability of errors in large-scale replacement logic, thereby reducing the occurrence of production accidents.

[0148] Based on the same inventive concept, this invention also provides a file anti-corruption system based on a distributed platform. Since the principle of this system in solving the problem is similar to that of the file anti-corruption method based on a distributed platform, the implementation of this system can refer to the implementation of the method, and the repeated parts will not be described again.

[0149] Figure 12 This is a schematic diagram of a file anti-corrosion system based on a distributed platform, as described in an embodiment of the present invention. Figure 12 As shown, the file corruption prevention system based on a distributed platform includes:

[0150] The file anti-corrosion device based on the distributed platform described above is applied to the platform; wherein, the platform includes a data integration module 1, an open platform server cluster 2, and a data lake 4.

[0151] The host is used to send host files to the file preservation device;

[0152] Downstream applications are used to receive the anti-corrosion documents pushed by the document anti-corrosion device.

[0153] like Figure 12 As shown, this solution, compared with existing technologies, focuses on ensuring complete consistency of upstream and downstream data processing during the mainframe process within the bank's core system, guaranteeing the smooth operation of business files, and employing different anti-corrosion processes based on the characteristics of the bank's core system and different stages of the mainframe process. By using an anti-corrosion layer for multiple data sources during the data storage process, it ensures consistency between the final stored data and single-data scenarios, guaranteeing the content and timeliness of downstream files. It includes a data integration module 1, an open platform server cluster 2, a mainframe 3, a data lake 4, and downstream applications 5. Specifically, the data integration module 1 is responsible for generating files from transaction data, reprocessing and distributing them; the open platform server cluster 2 handles the distributed system business processing on the existing platform; the mainframe 3 is the mainframe data processing center; the data lake 4 is responsible for storing and reprocessing files provided by the open platform server cluster 2 and the mainframe 3; and the downstream application 5 is the final application using the data.

[0154] Figure 13 This is a structural block diagram of the host in an embodiment of the present invention. For example... Figure 13As shown, mainframe 3 contains data routed through the mainframe, basic data running on the core mainframe, online batch services, and corresponding batch transmission processing data. The mainframe performs batch processing operations (incremental and storage merging) based on service needs and the output of the mainframe interface module 31, storing and integrating the data. Simultaneously, based on the output of the service processing module 32, it forms the basic data for the storage processing module 33. This basic data covers the online batch logic and data storage of services running on the core mainframe, providing data support for corresponding anti-corrosion processing requirements.

[0155] In practical implementation, the host interface module 31 defines basic data such as host card account data and transaction details, while the business processing module 32 defines the business logic relationships between the basic data, providing a reasonable logical reference for subsequent batch processing and anti-corrosion processing. By initiating specific online or batch processing in the business processing module 32, the storage processing module 33 is updated, and the batch processing module 34 downloads and updates the batch files to the data lake 4.

[0156] In summary, the file anti-corrosion system based on a distributed platform in this invention can effectively solve the problem of distributing files from a single host data source to multiple platforms and time periods. This shields downstream data users from large-scale replacement and modification, effectively reducing the workload of downstream replacement and the probability of errors in large-scale replacement logic, reducing the occurrence of production accidents, avoiding unnecessary data anomalies, and ensuring data validity.

[0157] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0158] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0159] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0160] While this specification provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded.

[0161] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0162] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0163] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0166] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0167] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0168] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0169] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0171] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0172] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. A file corruption prevention method based on a distributed platform, characterized in that, include: Obtain platform files, host files, and runtime phases; wherein, the runtime phases include a host-based phase, a dual-machine parallel platform-based phase, and a break-and-write phase; A verification file is generated based on the platform file, the host file, and the runtime phase; According to the aforementioned operational phase, the corresponding anti-corrosion script is selected to perform anti-corrosion on the verification document, generating an anti-corrosion document; wherein, the anti-corrosion script refers to a computer program or processing logic that is pre-made according to the operational phase and is used to perform specific processing on the verification document so that the generated anti-corrosion document is consistent with the original single host data source; the anti-corrosion document is consistent with the original single host data source in terms of content, structure and timeliness, thereby shielding the differences between multiple data sources; The anti-corrosion file is pushed to downstream applications; The step of generating a verification file based on the platform file, the host file, and the runtime phase includes: Compare the platform file and the host file to generate a comparison result; The number of platform verification files and the number of host verification files are determined based on the comparison results and the operational phase. The platform verification file is determined based on the number of platform files and platform verification files, and the host verification file is determined based on the number of host files and host verification files.

2. The file corruption prevention method based on a distributed platform according to claim 1, characterized in that, The comparison results generated by comparing the platform file and the host file include: The comparison method is determined based on the structure of the platform file and the structure of the host file; The platform file and the host file are compared according to the comparison method to generate the comparison result.

3. The file corruption prevention method based on a distributed platform according to claim 1, characterized in that, Also includes: Receive customer business instructions and insert corresponding records into the log according to the transaction type in the customer business instructions; Verify the customer's business instructions, and generate the platform file based on the verification results and the records.

4. A file preservation device based on a distributed platform, characterized in that, include: The acquisition module is used to acquire platform files, host files, and runtime phases; wherein, the runtime phases include a host-based phase, a dual-machine parallel platform-based phase, and a break-and-write phase; The verification file module is used to generate a verification file based on the platform file, the host file, and the runtime phase. The anti-corrosion module is used to select the corresponding anti-corrosion script according to the running stage to perform anti-corrosion on the verification file and generate an anti-corrosion file; wherein, the anti-corrosion script refers to a computer program or processing logic that is pre-made according to the running stage and is used to perform specific processing on the verification file so that the generated anti-corrosion file is consistent with the original single host data source; the anti-corrosion file is consistent with the original single host data source in terms of content, structure and timeliness, thereby shielding the differences between multiple data sources; The file push module is used to push the anti-corrosion file to downstream applications; The step of generating a verification file based on the platform file, the host file, and the runtime phase includes: Compare the platform file and the host file to generate a comparison result; The number of platform verification files and the number of host verification files are determined based on the comparison results and the operational phase. The platform verification file is determined based on the number of platform files and platform verification files, and the host verification file is determined based on the number of host files and host verification files.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the file anti-corrosion method based on a distributed platform as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the file anti-corrosion method based on a distributed platform as described in any one of claims 1 to 3.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the file anti-corrosion method based on a distributed platform as described in any one of claims 1 to 3.

8. A file corruption prevention system based on a distributed platform, characterized in that, include: The file anti-corrosion device based on a distributed platform as described in claim 4 is applied to the platform; The host is used to send host files to the file preservation device; Downstream applications are used to receive the anti-corrosion documents pushed by the document anti-corrosion device.

Images