A method and system for encrypted bid file decryption distribution

Abstract

The application discloses a kind of encrypted bidding file decryption distribution method and system, method includes submitting decryption task, and decryption task is confirmed and is sent out;Start decryption task, build FTP connection cache pool;Obtain encrypted file, and the encrypted file is decrypted;The file after decryption is copied to network card by Socket buffer in the form of binary stream;Dynamic acquisition distribution target information, and the decryption file is distributed.The FTP connection pool of the application is created once when service starts, and the cache mode of available connection effectively avoids the problem of repeated creation of concurrent decryption starting file acquisition request, saves the resource consumption of each creation time;The application does not write into disk after file decryption, effectively reduces the IO time consumption of system;Meanwhile, the application dynamically configures write target information, each file decryption will be delivered to specific target, and each address only distributes the effective file currently used, improves decryption transmission efficiency.

Description

Technical Field

[0001] This invention relates to the field of bid document decryption technology, and in particular to a method and system for decrypting and distributing encrypted bid documents. Background Technology

[0002] The bidding project is crucial to the company's business operations and urgently requires the use of information technology to conduct electronic bidding and procurement. Since the bidding process demands rapid completion of document decryption, transmission, and distribution within a limited timeframe, existing decryption technologies are distributed multi-process with a decryption speed of approximately 95GB / hour. For batch project bid documents exceeding 300GB, decryption alone takes more than 4 hours, which is insufficient and severely impacts the actual on-site bid review process. Therefore, research is being conducted on a special technology for decryption based on high-density computing. Summary of the Invention

[0003] The present invention aims to overcome the problem that the slow decryption speed of existing tender documents seriously affects the actual on-site tender review work, and provides a method and system for decrypting and distributing encrypted tender documents.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A method for decrypting and distributing encrypted tender documents includes the following steps: S1: Submitting a decryption task, confirming and dispatching the decryption task; S2: Starting the decryption task and building an FTP connection cache pool; S3: Obtaining the encrypted file and decrypting the encrypted file; S4: Copying the decrypted file to the network card in binary stream form from the Socket buffer; S5: Dynamically obtaining distribution target information and distributing the decrypted file according to the distribution target information. This invention's method for decrypting and distributing encrypted tender documents creates the FTP connection pool once at service startup, using a cached connection method to effectively avoid the problem of repeatedly creating requests for obtaining the initial decryption file during concurrent decryption, saving resource consumption during each creation, and significantly improving connection reuse and performance. After decryption, the encrypted file is not written to disk but transmitted via the FTP channel, effectively reducing system IO time. Simultaneously, this invention dynamically configures the writing target information; after each file is decrypted, it is delivered to a specific target, and each address only distributes the currently used valid file. Compared with the traditional indiscriminate writing design, this reduces the number of files transmitted, thereby improving decryption transmission efficiency.

[0006] As a preferred embodiment of the present invention, S1 specifically involves: the business application server initiating decryption task requests to the decryption host in batches, the decryption host confirming the decryption tasks, and dispatching decryption tasks to the decryption slaves.

[0007] In a preferred embodiment of the present invention, step S2 specifically involves: after receiving the decryption task, the slave device initiates the decryption task in parallel, constructing an FTP connection cache pool. The FTP connection cache pool establishes a minimum number of connections when the decryption program starts by default, maintains a specific configured number of active connections throughout the decryption process, and ultimately strictly limits the maximum number of connections. Each FTP connection in a single-machine decryption thread is obtained from the FTP connection cache pool, and the connection is released back to the pool after use. The connection pool is created once upon service startup, and the caching of available connections effectively avoids the problem of repeated creation of requests for concurrent decryption start file acquisition, saving resource consumption during each creation, and significantly improving connection reuse and performance.

[0008] As a preferred embodiment of the present invention, S3 specifically involves: the decryption slave device obtaining the encrypted file from the file storage server via FTP and decrypting the encrypted file.

[0009] As a preferred embodiment of the present invention, S4 specifically involves: after the decryption slave device decrypts the encrypted file, the decrypted file is not written to the disk, but directly copied from memory as a binary stream from the Socket buffer to the network card, and then transmitted via the FTP channel. A zero-copy pre-read mechanism is implemented: mmap+write replaces the traditional read+write operation, reducing one CPU copy operation. After the DMA controller copies data from the hard drive to the read buffer, it maps the address of the read buffer to the address of the user buffer. The kernel buffer and application buffer share the same address, thereby reducing one CPU copy from the read buffer to the user buffer. After the decryption slave device decrypts the acquired encrypted file, it does not write it to the disk, but directly copies it from memory as a binary stream from the Socket buffer to the network card, and then transmits it via the FTP channel, effectively reducing system I / O time.

[0010] As a preferred embodiment of the present invention, S5 specifically involves: the decryption slave dynamically acquiring distribution target information and distributing the decrypted files to each evaluation site server according to the distribution target information. The distribution target information specifically refers to the specific distribution address information of all individual decrypted files. Dynamically configuring the writing target information completely overturns the previous rigid situation of writing all data to multiple addresses. In terms of business, each site address only needs a portion of the data, and the distributed decryption server operates in a service-sharing mode. Therefore, the previous solution adopted a full-volume, indiscriminate writing design, which resulted in significant additional overhead in terms of the number of files written and network transmission. After dynamic acquisition, each decrypted file is delivered to a specific target, and each address only distributes the currently used valid files. Compared with the traditional indiscriminate writing design, this reduces the number of files transmitted, thereby improving decryption transmission efficiency.

[0011] As a preferred embodiment of the present invention, the method further includes writing back the file decryption status. After the decryption slave device completes the decryption of the encrypted file, it initiates a decryption status write-back to the decryption host. The decryption host confirms the decryption result of the decryption slave device, and the business application server obtains the decryption result from the decryption host.

[0012] An encrypted bid document decryption and distribution system includes: a business application server that initiates decryption task requests to a decryption host and obtains decryption results; a file storage server that stores encrypted files; a decryption host that confirms the decryption task and decryption results and distributes the decryption task to decryption slaves; decryption slaves that execute decryption tasks, distribute decrypted files, and initiate decryption status write-back; and a bid evaluation site server that receives decrypted files.

[0013] Therefore, the present invention has the following beneficial effects: The encrypted tender document decryption and distribution method and system of the present invention creates an FTP connection pool once at service startup, and the available connection caching method effectively avoids the problem of repeated creation of requests for obtaining the initial file for concurrent decryption, saving resource consumption during each creation, and significantly improving connection reuse and performance; after decryption, the present invention does not write the encrypted file to the disk, but transmits it through the FTP channel, effectively reducing system IO time; at the same time, the present invention dynamically configures the writing target information, and after each file is decrypted, it is delivered to a specific target. Each address only distributes the currently used valid files, reducing the number of files transmitted compared to the traditional indiscriminate writing design, thereby improving decryption transmission efficiency. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention;

[0015] Figure 2 This is a prior art decryption flowchart in the embodiments;

[0016] Figure 3 This is a prior art decryption timing diagram in the embodiments;

[0017] Figure 4 This is a new decryption flowchart in an embodiment of the present invention;

[0018] Figure 5 This is a new decryption timing diagram in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0020] like Figure 1 As shown, a method for decrypting and distributing encrypted tender documents includes the following steps:

[0021] S1: Submit decryption tasks, confirm and dispatch decryption tasks; S1 specifically: The business application server initiates decryption task requests to the decryption host in batches, the decryption host confirms the decryption tasks, and dispatches the decryption tasks to the decryption slaves.

[0022] S2: Start the decryption task and build an FTP connection cache pool; S2 specifically means: after receiving the decryption task, the decryption slave machine starts the decryption task in parallel and builds an FTP connection cache pool. The FTP connection cache pool establishes a minimum number of connections when the decryption program is started by default, and maintains a specific configured number of active connections throughout the decryption process, and finally strictly limits the maximum number of connections. Each FTP connection in a single decryption thread is obtained from the FTP connection cache pool, and the connection is released back to the connection pool after use.

[0023] S3: Obtain the encrypted file and decrypt it; S3 specifically means: the slave device obtains the encrypted file from the file storage server via FTP and decrypts it.

[0024] S4: Copy the decrypted file to the network card in binary stream form from the Socket buffer; S4 specifically: after the decryption slave device decrypts the encrypted file, the decrypted file is not written to the disk, but is directly copied from memory to the network card in binary stream form from the Socket buffer and transmitted through the FTP channel.

[0025] S5: Dynamically acquire distribution target information and distribute the decrypted files according to the distribution target information; S5 specifically means: the decryption slave dynamically acquires the distribution target information and distributes the decrypted files to each bidding site server according to the distribution target information, wherein the distribution target information specifically refers to the specific distribution address information of all individual decrypted files.

[0026] It also includes file decryption status write-back. After the decryption slave device completes the decryption of the encrypted file, it initiates a decryption status write-back to the decryption master device. The decryption master device confirms the decryption result of the decryption slave device, and the business application server obtains the decryption result from the decryption master device.

[0027] This invention discloses a method for decrypting and distributing encrypted tender documents. The FTP connection pool is created all at once during service startup, and the available connections are cached, effectively avoiding the problem of repeated creation of requests for obtaining the initial decryption file during concurrent decryption. This saves resource consumption during each creation, significantly improving connection reuse and performance. After decryption, the encrypted file is not written to disk but transmitted via the FTP channel, effectively reducing system I / O time. Furthermore, this invention dynamically configures the writing target information; each decrypted file is delivered to a specific target, and each address only distributes the currently used valid files. Compared to traditional indiscriminate writing designs, this reduces the number of files transmitted, thereby improving decryption and transmission efficiency.

[0028] This invention also proposes an encrypted bid document decryption and distribution system, applicable to an encrypted bid document decryption and distribution method of this invention, comprising: a business application server: initiating a decryption task request to a decryption host and obtaining decryption results; a file storage server: storing encrypted files; a decryption host: confirming the decryption task and decryption results, and distributing the decryption task to decryption slaves; decryption slaves: executing the decryption task, distributing decrypted files, and initiating decryption status write-back; and an evaluation site server: receiving decrypted files.

[0029] In this embodiment, a method for decrypting and distributing encrypted tender documents according to the present invention is further described.

[0030] Because the primary goal of uploading bid documents to the bidding system is to ensure the security and convenience of the bid evaluation process, suppliers encrypt and sign their bids online using their unique USB tokens on the electronic procurement platform before uploading them to the system's backend file server. After the project bids are opened, the encrypted files need to be decrypted within a certain timeframe (usually within 4 hours) and distributed to a designated bid evaluation server for experts to review.

[0031] Due to the inherent confidentiality requirements of the bidding process and the need for isolation at the bidding site, existing decryption solutions involve a distributed deployment of independent decryption platforms, with online real-time task scheduling for file decryption and real-time distribution of decrypted files. While the current platform architecture can meet the security and distribution needs of bid documents in an unattended environment throughout the entire electronic bidding process, its decryption speed is unsatisfactory, especially in large bidding batches where encrypted files can reach 300-500GB, with decryption times reaching 4-5 hours, severely impacting the actual on-site bid review process. Figure 2 and Figure 3 The diagram shown is a flowchart and timing diagram of the old decryption process in the existing technology.

[0032] The current old online decryption process includes the following steps:

[0033] The business system submits a specified batch of decryption tasks to the distributed decryption platform.

[0034] The distributed decryption platform receives the decryption task, confirms it, and replies to the business system.

[0035] The decryption platform dispatches decryption tasks.

[0036] The decryption slave receives the dispatched decryption task and retrieves the encrypted file via FTP.

[0037] The slave device decrypts the acquired encrypted file.

[0038] The decryption slave device will then distribute the decrypted files.

[0039] Through monitoring and tracking of the above decryption process, it was found that in a company's internal 1000M bandwidth network, a single decryption server configured with 32C, 64G, and 200SSD resources, takes approximately 2 seconds to complete the decryption of a single encrypted file of about 100MB. The percentage of time consumed by each process is as follows:

[0040] FTP connection establishment 20%;

[0041] 35% of encrypted files were obtained;

[0042] File decryption and status rewrite: 10%;

[0043] 35% of decrypted files were returned via a single path.

[0044] Therefore, it can be seen that the time consumption in the file decryption process mainly lies in the network file transfer process, followed by the establishment of the connection request. The file decryption itself takes relatively little time. Therefore, in order to improve the decryption speed, it is necessary to innovate and optimize the connection establishment and network transmission. Based on this, the method of the present invention was developed.

[0045] like Figure 4 and Figure 5 The diagram shown is an innovative decryption flowchart and timing diagram of the present invention. The technical solution of the present invention is innovative in the following aspects:

[0046] FTP request connection cache pool construction and connection caching: By configuring a minimum number of connections to be established when the decryption program starts by default, specifically 10 in this embodiment, and maintaining a specific configured number of active connections during the decryption process, specifically 5 in this embodiment, and finally strictly limiting the maximum number of connections, specifically 20 in this embodiment, each FTP connection in the single-machine decryption thread is obtained from the connection cache pool. After use, the connection is not closed, but released back to the connection pool, thereby solving the time-consuming problem of frequent connection establishment.

[0047] Zero-copy file read-ahead mechanism: This mechanism uses mmap+write instead of the traditional read+write operation, reducing one CPU copy operation. After the DMA controller copies data from the hard drive to the read buffer, it maps the address of the read buffer to the address of the user buffer. The kernel buffer and application buffer share the same address, thus reducing one CPU copy from the read buffer to the user buffer. When decrypting an encrypted file obtained from a slave device, it is not written to disk but directly copied from memory as a binary stream via the socket buffer to the network card and then transmitted through the FTP channel, effectively reducing system I / O time.

[0048] Dynamic destination distribution: Since there are multiple decryption file distribution addresses, a Map configuration is added to dynamically obtain information such as the specific distribution address of a single decryption file, avoiding the previous rigid processing that delivered each destination indiscriminately, effectively shortening the overall network transmission time.

[0049] To address the issues of repeated connection request creation and long network transmission times in the old solution, the innovative aspects of this invention have the following significant advantages:

[0050] The connection pool is created once when the service starts, and the available connections are cached, which effectively avoids the problem of repeated creation of connections for concurrent decryption of the starting file. This saves resources when creating connections each time, and significantly improves connection reuse and performance.

[0051] The use of zero-copy file caching technology saves one CPU copy through mmap. At the same time, since the memory in the user process is virtual and only mapped to the kernel's read buffer, it can save half of the memory space, making it more suitable for the transfer of large files.

[0052] Dynamic configuration of target information writing completely overturns the previous rigid approach of writing all data to multiple addresses. In business operations, each site address only needs a portion of the data, and the distributed decryption server operates in a service-sharing mode. Therefore, the previous solution used a full, indiscriminate writing design, which resulted in significant additional overhead in terms of the number of files written and network transmission. With dynamic acquisition, each decrypted file is delivered to a specific target, and each address only distributes the currently used valid files. Compared to the traditional indiscriminate writing design, this reduces the number of files transmitted, thereby improving decryption and transmission efficiency.

[0053] The facts have proven the outstanding effect of the new solution of this invention. After the transformation of the new solution, the efficiency of file decryption and distribution has been greatly improved, from 95G / hour of the old solution to 250G / hour, which has shortened the decryption time by a full 60%.

[0054] Terminology Explanation:

[0055] Connection pool: Instead of directly connecting to the server, the client stores pre-created connections in a pool. When a request comes in, the client directly uses the pre-created connection to access the database.

[0056] Mmap: Maps the address of the read buffer to the address of the user buffer. The kernel buffer and the application buffer are shared, thereby reducing one CPU copy from the read buffer to the user buffer.

[0057] DMA: Direct Memory Access technology. Essentially, it is a separate chip on the motherboard that is used to transfer data between memory and I / O devices, thereby reducing CPU wait time.

[0058] Zero-copy: Zero-copy technology refers to a technique where the CPU does not need to copy data from one memory location to another before performing an operation. This technique is often used to save CPU cycles and memory bandwidth when transferring files over a network.

[0059] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions conceived without creative effort should be included within the scope of protection of the present invention.

Claims

1. A method for decrypting and distributing encrypted tender documents, characterized in that, Includes the following steps: S1: Submit the decryption task, confirm and dispatch the decryption task; S2: Start the decryption task and build an FTP connection cache pool. Each FTP connection in the single-machine decryption thread is obtained from the FTP connection cache pool. The FTP connection cache pool is configured to establish the minimum number of connections when the decryption program is started by default. S3: Obtain the encrypted file, decrypt the encrypted file, and do not write the decrypted file to the disk; S4: Copy the decrypted file to the network card in binary stream form from the Socket buffer; S5: Dynamically acquire distribution target information. After dynamic acquisition, each file will be delivered to a specific target after decryption. The decrypted files will be distributed according to the distribution target information.

2. The method for decrypting and distributing encrypted tender documents according to claim 1, characterized in that, Specifically, S1 involves the business application server initiating decryption task requests to the decryption host in batches, the decryption host confirming the decryption tasks, and dispatching decryption tasks to the decryption slaves.

3. The method for decrypting and distributing encrypted tender documents according to claim 1, characterized in that, S2 specifically refers to: after receiving the decryption task, the decryption slave device starts the decryption task in parallel, builds an FTP connection cache pool, establishes a minimum number of connections when the decryption program is started by default, maintains a specific configured number of active connections during the decryption process, and finally strictly limits the maximum number of connections. Each FTP connection in the single-machine decryption thread is obtained from the FTP connection cache pool, and the connection is released back to the connection pool after use.

4. The method for decrypting and distributing encrypted tender documents according to claim 1, characterized in that, Specifically, S3 involves the decryption slave device retrieving the encrypted file from the file storage server via FTP and decrypting the encrypted file.

5. The method for decrypting and distributing encrypted tender documents according to claim 1, characterized in that, Specifically, S4 is as follows: After the decryption slave device decrypts the encrypted file, the decrypted file is not written to the disk, but is directly copied from memory in the form of a binary stream from the Socket buffer to the network card, and then transmitted through the FTP channel.

6. The method for decrypting and distributing encrypted tender documents according to claim 1, characterized in that, S5 specifically involves: dynamically acquiring distribution target information from the decryption slave device, and distributing the decrypted files to each bidding site server according to the distribution target information. The distribution target information specifically refers to the specific distribution address information of all individual decrypted files.

7. A method for decrypting and distributing encrypted tender documents according to any one of claims 1-6, characterized in that, It also includes file decryption status write-back. After the decryption slave device completes the decryption of the encrypted file, it initiates a decryption status write-back to the decryption master device. The decryption master device confirms the decryption result of the decryption slave device, and the business application server obtains the decryption result from the decryption master device.

8. A system for decrypting and distributing encrypted bid documents, applicable to the method for decrypting and distributing encrypted bid documents as described in any one of claims 1-7, characterized in that, include: Business application server: Initiates decryption task requests to the decryption host and obtains decryption results; File storage server: stores encrypted files; Decryption host: Confirms the decryption task and decryption result, and distributes the decryption task to the decryption slave; Decryption slave device: Executes decryption tasks, distributes decryption files, and initiates decryption status write-back; On-site server for bid evaluation: Receives decrypted files.

Images