Batch data import method, device and equipment based on stream load protocol, medium and product

Abstract

The invention discloses a batch data importing method, device and equipment based on a stream load protocol, a medium and a product. The method comprises the following steps: in response to a batch data import request, determining batch data and an import data volume; if the imported data volume reaches a preset data volume threshold value and a transaction identifier acquired from the database server side in advance exists, generating an HTTP push message based on a stream load protocol according to a pre-stored hyper text transfer protocol HTTP connection related parameter, the transaction identifier and the batch data; feeding back the HTTP push message to the database server, so that the database server analyzes the HTTP push message to obtain a transaction identifier and batch data, imports the batch data, and generates and feeds back a batch data import response result; and generating a communication connection closing request, and sending the communication connection closing request to the database server, so that the database server closes the HTTP service process bound with the driver instance corresponding to the database driver.

Description

Technical Field

[0001] This invention relates to the field of batch data import from databases, and more particularly to a method, apparatus, device, medium, and product for batch data import based on the stream load protocol. Background Technology

[0002] As the core carrier of data storage and management, databases are crucial for the stable operation of various data processing businesses through the efficient and secure import of large volumes of data. With the expansion of business scale, the need to import batches of data into databases is becoming increasingly urgent, placing higher demands on the timeliness and transaction consistency of the import process.

[0003] Existing database batch import solutions have significant limitations: they lack optimized transmission protocols and flexible transaction control mechanisms, which can easily lead to data loss, duplicate imports, and other problems, affecting data consistency. Improper management of the timing of HTTP (Hypertext Transfer Protocol) service activation can result in resource waste or transmission delays. Non-standard parameter interaction between the driver and server can easily lead to connection failures or data security risks. Furthermore, they do not fully adapt to the synchronization requirements between distributed database nodes, resulting in low cross-node transmission efficiency. Summary of the Invention

[0004] This invention provides a method, apparatus, device, medium, and product for batch data import based on the stream load protocol, in order to optimize the process of batch importing database data, thereby ensuring the execution efficiency and stability of batch importing database data.

[0005] According to one aspect of the present invention, a batch data import method based on the stream load protocol is provided, applied to a database driver, the method comprising:

[0006] In response to a batch data import request, determine the batch data and the amount of data to be imported based on the batch data import request;

[0007] If the amount of imported data reaches a preset data volume threshold and there is a transaction identifier obtained from the database server in advance, then an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, the transaction identifier, and the batch data.

[0008] The HTTP push message is sent back to the database server so that the database server can parse the HTTP push message to obtain the transaction identifier and batch data. The transaction processing batch is determined according to the transaction identifier, and the batch data is imported under the transaction processing batch. The corresponding batch data import response result is generated and fed back.

[0009] Based on the batch data import response result, a communication connection closure request is generated and sent to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver.

[0010] According to another aspect of the present invention, a batch data import method based on the stream load protocol is provided, applied to a database server, comprising:

[0011] Retrieve HTTP push messages from the database driver;

[0012] The HTTP push message is parsed to obtain the transaction identifier and batch data;

[0013] Based on the transaction identifier, the transaction batch is determined, and the batch data is imported under the transaction batch. The batch data import response result under the corresponding transaction batch is generated and fed back, so that the database driver can generate a communication connection closure request based on the batch data import response result and send the communication connection closure request to the database server.

[0014] Shut down the HTTP service process bound to the driver instance corresponding to the database driver;

[0015] The HTTP push message is generated in the following way:

[0016] In response to a batch data import request, the database driver determines the batch data and the amount of data to be imported based on the batch data import request. When it is determined that the amount of data to be imported reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server, an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, the transaction identifier, and the batch data.

[0017] The HTTP connection-related parameters are generated and fed back by the database server after the database driver and the database server successfully establish a communication connection; after the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started.

[0018] According to another aspect of the present invention, a batch data import device based on the stream load protocol is provided, configured on a database driver, the device comprising:

[0019] The data import module, in response to a batch data import request, determines the batch data and the amount of data to be imported based on the batch data import request;

[0020] The HTTP push message generation module is used to determine if the amount of imported data reaches a preset data volume threshold and there is a transaction identifier obtained from the database server in advance. Then, based on the pre-stored Hypertext Transfer Protocol (HTTP) connection-related parameters, the transaction identifier, and the batch data, an HTTP push message is generated according to the stream load protocol.

[0021] The response result generation module is used to send the HTTP push message back to the database server, so that the database server can parse the HTTP push message to obtain the transaction identifier and batch data, determine the transaction processing batch according to the transaction identifier, and perform import operation on the batch data under the transaction processing batch, and generate and send back the batch data import response result under the corresponding transaction processing batch.

[0022] The request generation module is used to generate a communication connection closure request based on the batch data import response results, and send the communication connection closure request to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver.

[0023] According to one aspect of the present invention, a batch data import device based on the stream load protocol is provided, configured on a database server, the device comprising:

[0024] The HTTP push message acquisition module is used to acquire HTTP push messages from the database driver.

[0025] The HTTP push message parsing module is used to parse the HTTP push message to obtain the transaction identifier and batch data;

[0026] The batch data import response result generation module is used to determine the transaction processing batch based on the transaction identifier, perform the batch data import operation under the transaction processing batch, generate and return the batch data import response result under the corresponding transaction processing batch, so that the database driver can generate a communication connection closure request based on the batch data import response result and send the communication connection closure request to the database server.

[0027] The HTTP service process shutdown module is used to shut down the HTTP service process bound to the driver instance corresponding to the database driver.

[0028] The HTTP push message determination module is used by the database driver to respond to a batch data import request, determine the batch data and the amount of data to be imported based on the batch data import request, and when it is determined that the amount of data to be imported reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server, generate an HTTP push message based on the stream load protocol according to the pre-stored Hypertext Transfer Protocol (HTTP) connection-related parameters, the transaction identifier and the batch data.

[0029] The HTTP connection-related parameters are generated and fed back by the database server after the database driver and the database server successfully establish a communication connection; after the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started.

[0030] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0031] At least one processor; and

[0032] A memory that is communicatively connected to at least one processor; wherein,

[0033] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor can execute a batch data import method based on the stream load protocol according to any embodiment of the present invention.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute a batch data import method based on a stream load protocol according to any embodiment of the present invention.

[0035] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements a batch data import method based on a stream load protocol according to any embodiment of the present invention.

[0036] The technical solution of this invention responds to a batch data import request, determines the batch data and the amount of data to be imported based on the request, and if the amount of data to be imported reaches a preset data volume threshold and a transaction identifier obtained in advance from the database server exists, then an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, the transaction identifier, and the batch data. After the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started, and the HTTP push message is fed back to the database server so that the database server can parse the HTTP push message to obtain the transaction identifier and the batch data, determine the transaction processing batch based on the transaction identifier, and perform the import operation on the batch data under the transaction processing batch, generate and feed back the batch data import response result under the corresponding transaction processing batch, generate a communication connection closure request based on the batch data import response result, and send the communication connection closure request to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver. The above technical solutions can significantly improve the efficiency and reliability of batch data import in databases. On the one hand, by reducing the number of data transmission interactions through the stream load protocol, large amounts of data can be imported quickly. On the other hand, the atomicity of data import is guaranteed by transaction identifiers, avoiding data loss or duplicate import issues and ensuring database data consistency. At the same time, the standardized parameter interaction between the driver and the server and the precise control of the HTTP service process reduce the risk of connection failure, optimize system resource usage, and meet the core needs of various businesses for high efficiency and high security in batch data import in databases.

[0037] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of a batch data import method based on the stream load protocol according to Embodiment 1 of the present invention;

[0040] Figure 2 This is a flowchart of a batch data import method based on the stream load protocol according to Embodiment 2 of the present invention;

[0041] Figure 3A This is an interaction diagram between the database server and the database driver in a batch data import method based on the stream load protocol provided in Embodiment 3 of the present invention.

[0042] Figure 3B This is an interaction diagram between the database server and the database driver for another batch data import method based on the stream load protocol provided in Embodiment 3 of the present invention.

[0043] Figure 3C This is an interaction diagram between the database server and the database driver for another batch data import method based on the stream load protocol provided in Embodiment 3 of the present invention.

[0044] Figure 4 This is a schematic diagram of the structure of a batch data import device based on the stream load protocol that can be applied to the database driver end, provided in Embodiment 4 of the present invention;

[0045] Figure 5 This is a schematic diagram of a batch data import device based on the stream load protocol, which can be applied to a database server, according to Embodiment 4 of the present invention.

[0046] Figure 6 This is a schematic diagram of the structure of an electronic device that implements a batch data import method based on the stream load protocol according to an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] Example 1

[0050] Figure 1 This is a flowchart illustrating a batch data import method based on the stream load protocol provided in Embodiment 1 of the present invention. This embodiment is applicable to application scenarios involving the efficient import of large amounts of data into a database. The method can be executed by a batch data import device based on the stream load protocol, applicable to the database driver. This batch data import device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, this method is applied to the database driver side and includes:

[0051] S101. In response to the batch data import request, determine the batch data and the amount of data to be imported based on the batch data import request.

[0052] S102. If the amount of imported data reaches the preset data volume threshold and there is a transaction identifier obtained from the database server in advance, then an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, transaction identifier and batch data.

[0053] HTTP connection-related parameters are generated and fed back by the database server after the database driver and database server successfully establish a communication connection; after the database server and database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started.

[0054] S103. The HTTP push message is sent back to the database server so that the database server can parse the HTTP push message to obtain the transaction identifier and batch data. The transaction processing batch is determined according to the transaction identifier, and the batch data is imported under the transaction processing batch. The corresponding batch data import response result is generated and fed back.

[0055] S104. Based on the batch data import response results, generate a communication connection closure request and send the communication connection closure request to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver.

[0056] A batch data import request can be an instruction initiated by a user application to write a specified dataset to the database in batches. The batch data can be multiple structured data entries conforming to the database table structure definition. The import volume can be the number of data records contained in the batch data.

[0057] For example, the database driver receives a batch data import request initiated by a user application, extracts the data range and data type contained in the request, and determines the total amount of data to be imported in the batch dataset. For instance, a bank system imports 100,000 user financial product purchase records into the database. The user application sends a batch data import request to the database driver. After receiving the request, the driver extracts the transaction records carried in the request as batch data and calculates that the amount of data to be imported is 100,000 records.

[0058] Furthermore, in order to establish the communication foundation in advance and obtain the parameters required for subsequent data transmission, before responding to a batch data import request and determining the batch data and the amount of data to be imported based on the batch data import request, the following steps are also included:

[0059] A communication connection establishment request is initiated to the database server so that the database server can establish a communication connection based on the communication connection establishment request. At the same time, the HTTP service process bound to the driver instance corresponding to the database driver is started. After the communication connection is successfully established, the relevant HTTP connection parameters are determined, and a connection establishment success message is generated and sent back based on the relevant HTTP connection parameters.

[0060] In this system, the database server can be a server that provides data storage and processing services. The connection establishment request can be a connection request instruction sent from the driver to the server, containing authentication information. The HTTP service process can be a running process on the server bound to the driver instance that handles HTTP requests. HTTP connection parameters can include the request port, authentication token, encryption key, etc., required for communication. The driver instance can be a running instance exclusively allocated by the server for the current driver, used for handling data interaction. The connection establishment success message can be a message sent back by the server, carrying HTTP connection parameters.

[0061] For example, the database driver sends a communication connection establishment request containing the driver identifier and user authentication information to the database server. After the database server verifies the request, it starts the driver instance corresponding to the database driver and the bound HTTP service process, determines the HTTP connection parameters such as the request port 8080 and the encryption key key7, and generates and sends back a connection establishment success message containing these parameters.

[0062] The preset data volume threshold can be the amount of data that triggers the batch import protocol, pre-set by relevant technical personnel. The transaction identifier can be a unique transaction ID assigned by the database server. The stream load protocol can be a protocol that supports efficient transmission of large amounts of data. The HTTP push message can be an HTTP PUT request conforming to this protocol standard. The database driver can be an intermediary component connecting the user application and the database server. The HTTP PUT request can be the carrier for batch data transmission using the stream load protocol.

[0063] For example, the database driver determines whether the imported data volume has reached a preset data volume threshold, and simultaneously verifies whether it has obtained a valid transaction identifier from the database server. If both conditions are met, it retrieves the relevant HTTP connection parameters stored locally, integrates the parameters, the transaction identifier, and the batch data, and generates a corresponding HTTP push message following the message format requirements of the stream load protocol. For instance, if the preset data volume threshold is set to 10,000 records, and the database driver detects that 100,000 imported data records have reached the threshold, and has previously obtained the transaction identifier TID2001 from the server through a transaction ID request, the driver then retrieves the stored HTTP connection parameters such as port 8080 and token abc123xyz, combines the transaction identifier with the 100,000 batch data records, and generates an HTTP push message based on the stream load protocol.

[0064] Furthermore, to improve the batch data import method and enhance its applicability, the method also includes:

[0065] Step a1: If the amount of imported data reaches the preset data volume threshold and there is no transaction identifier obtained from the database server in advance, then an HTTP push message is generated based on the pre-stored HTTP connection-related parameters and the preset identifier, according to the stream load protocol; the preset identifier is carried in the tag field of the request header of the HTTP push message; the tag field is a field defined in the stream load protocol.

[0066] Step a2: The HTTP push message is sent back to the database server so that the database server can parse the HTTP push message to obtain the preset identifier and batch data. If it is determined from the preset identifier that there is a transaction commit request initiated by the database driver within the preset time period, the transaction processing batch is determined according to the transaction commit request, and the batch data is imported under the transaction processing batch. The corresponding batch data import response result is generated and fed back.

[0067] The preset identifier can be a field value used by the database driver and database server to identify the data source. The label field in the HTTP push message request header can be the label field defined by the stream load protocol, where the label field can be a field provided by the stream load protocol, and the field value can be preset by relevant technical personnel according to actual needs.

[0068] For example, if the database driver does not request a transaction identifier from the database server in advance, the preset identifier can be set to BATCH_001. When the amount of imported data reaches the preset data volume threshold, the database driver generates an HTTP push message based on the HTTP connection parameters and the preset identifier, and carries the preset identifier in the label field of the request header of the HTTP push message.

[0069] The preset time period can be a pre-defined time period. The transaction commit request can be a transaction start request initiated by the database driver. The transaction processing batch can be the transaction processing batch corresponding to the transaction start request. The batch data import response result can be the import status indicator and related details returned by the data server.

[0070] For example, if the preset time period is 30 minutes, after the database server parses the HTTP push message, it obtains the preset identifier BATCH_001 through the label field, identifies that the data comes from the database driver, detects that there is a Start Transaction instruction initiated by the database driver within 30 minutes and no Commit instruction has been received, determines that the batch data belongs to the transaction batch corresponding to the instruction, and reports a successful import result after the data import is completed, with a total of 100,000 records. The Start Transaction instruction and Commit instruction can be instructions initiated by the database driver to mark the start and commit status of the transaction, ensuring the atomicity of the transaction during the batch data import.

[0071] For example, the driver sends an HTTP push message to the server. After parsing the message, the server extracts the transaction identifier and batch data. It uses the identifier to locate the corresponding transaction processing batch, performs data writing operations under that batch, and generates a response result indicating successful import of 100,000 records, which is then fed back to the driver.

[0072] Among them, the communication connection closure request can be an instruction sent from the driver to the server, requesting the closure of the corresponding HTTP service process.

[0073] For example, after receiving a successful import response, the database driver generates a communication connection close request and sends it to the database server. If a failure response is received, a retransmission request is sent to the database server, requesting the database driver to re-push the batch data. Only when the data import is successful is the close request triggered, requiring the database server to close the HTTP service process bound to the current driver instance.

[0074] The technical solution of this invention can be applied to a database driver. In response to a batch data import request, the batch data and the amount of data to be imported are determined based on the request. If the amount of data to be imported reaches a preset data volume threshold and a transaction identifier previously obtained from the database server exists, an HTTP push message is generated based on the stream load protocol, according to pre-stored HTTP connection parameters, the transaction identifier, and the batch data. After the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started. The HTTP push message is fed back to the database server, allowing the database server to parse the message, obtain the transaction identifier and the batch data, determine the transaction processing batch based on the transaction identifier, and perform import operations on the batch data under the transaction processing batch. A batch data import response result is generated and fed back under the corresponding transaction processing batch. Based on the batch data import response result, a communication connection closure request is generated and sent to the database server, allowing the database server to close the HTTP service process bound to the driver instance corresponding to the database driver. The above technical solutions can significantly improve the efficiency and reliability of batch data import in databases. On the one hand, by reducing the number of data transmission interactions through the streamload protocol, large amounts of data can be imported quickly. On the other hand, the atomicity of data import is guaranteed by transaction identifiers, avoiding data loss or duplicate import issues and ensuring database data consistency. At the same time, the standardized parameter interaction between the driver and the server and the precise control of the HTTP service process reduce the risk of connection failure, optimize system resource usage, and meet the core needs of various businesses for high efficiency and high security in batch data import in databases.

[0075] Example 2

[0076] Figure 2 This is a flowchart illustrating a batch data import method based on the stream load protocol provided in Embodiment 2 of the present invention. This embodiment is applicable to application scenarios involving the efficient import of large amounts of data. The method can be executed by a batch data import device based on the stream load protocol, applicable to a database server. This batch data import device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 2 As shown, applied to the database server, this method includes:

[0077] S201. Obtain HTTP push messages from the database driver.

[0078] S202. Parse the HTTP push message to obtain the transaction identifier and batch data.

[0079] S203. Based on the transaction identifier, determine the transaction processing batch, and perform an import operation on the batch data under the transaction processing batch. Generate and return the batch data import response result under the corresponding transaction processing batch, so that the database driver can generate a communication connection closure request based on the batch data import response result and send the communication connection closure request to the database server.

[0080] S204. Close the HTTP service process bound to the driver instance corresponding to the database driver.

[0081] For example, the database server receives an HTTP push message from the database driver, which carries information such as transaction identifier, batch data, and HTTP connection-related parameters.

[0082] In one optional embodiment, the HTTP push message is generated as follows:

[0083] Step b1: In response to the batch data import request, the database driver determines the batch data and the amount of data to be imported based on the batch data import request. When it is determined that the amount of data to be imported reaches the preset data volume threshold and there is a transaction identifier obtained from the database server in advance, an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, transaction identifier and batch data.

[0084] Step b2: After the database driver and the database server successfully establish a communication connection, the HTTP connection-related parameters are generated and fed back by the database server. After the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started.

[0085] For example, after responding to a batch data import request, the database driver first determines the range and volume of the batch data to be imported. If it confirms that the imported data volume reaches a preset threshold and that the transaction identifier has been obtained, it retrieves pre-stored HTTP connection parameters and, combined with the transaction identifier and the batch data, generates an HTTP push message based on the stream load protocol. For instance, if the database driver receives a batch data import request from a user application, extracts 80,000 user loan application records as batch data, and calculates that the imported data volume is 80,000 records, which reaches the preset threshold of 10,000 records, and the database driver has previously obtained the transaction identifier TID2002 from the database server, then the database driver retrieves pre-stored HTTP connection parameters and, combined with the transaction identifier and the batch data, generates a corresponding HTTP push message based on the stream load protocol.

[0086] For example, after the database driver and the database server successfully establish a communication connection, the server generates and sends back HTTP connection-related parameters, and simultaneously starts the driver instance corresponding to the database driver, as well as the HTTP service process bound to that instance. For instance, after the database driver and the database server complete the communication connection establishment, the database server generates HTTP connection-related parameters including the request port, authentication token, and encryption key, and sends them back to the database driver. At the same time, the database server starts the driver instance corresponding to the database driver, as well as the HTTP service process bound to that driver instance.

[0087] The above technical solution constructs a standardized batch data import link through communication connection establishment, parameter feedback and data transmission process coordination. First, a dedicated secure communication channel is built and the HTTP service process bound to the driver instance is started as needed to avoid resource waste. Then, based on the HTTP connection related parameters and transaction identifiers, combined with the data volume threshold verification mechanism, a stream load protocol HTTP push message is generated. This not only enables the rapid transmission of large amounts of data with the help of the protocol, but also ensures the atomicity of data import, prevents data loss or duplicate import, and greatly improves import efficiency and reliability.

[0088] Furthermore, in order to achieve large-scale data synchronization between nodes within the database server, in an optional embodiment, the above method further includes:

[0089] Step c1: In response to the data synchronization request, determine the amount of data to be synchronized. If the amount of data to be synchronized reaches the preset data amount threshold, generate an HTTP push synchronization request based on the stream load protocol according to the data to be synchronized.

[0090] Step c2: Based on the data synchronization request, determine the target data node and send the HTTP push synchronization request to the target data node so that the target data node can determine the data to be synchronized based on the HTTP push synchronization request and perform data import operation on the data to be synchronized.

[0091] For example, each data node of the distributed database starts an HTTP service when it starts up. Its HTTP service parameters are stored in the core configuration database of the database system. When a data node responds to a data synchronization request, it first determines the amount of data to be synchronized. When the amount of data reaches a preset threshold, it constructs an HTTP push synchronization request that conforms to the requirements of the stream load protocol based on the data to be synchronized, fills in the fields specified in the protocol, and generates an HTTP push synchronization request.

[0092] The target data node can be the node that needs to be imported with synchronized data.

[0093] For example, the node initiating data synchronization determines the target data nodes based on the data synchronization request, sends the generated HTTP push synchronization request to the target nodes, and the target nodes, acting as HTTP servers, receive the request, parse the streamload protocol messages, extract the data to be synchronized, and then perform the data import operation. For instance, node A determines that the target data nodes are nodes B and C based on the data synchronization request, sends the generated HTTP push synchronization request to both target nodes B and C, and nodes B and C, acting as HTTP servers, receive the request, parse the streamload protocol messages, extract the 50,000 user deposit detail records to be synchronized, and then perform the data import operation to complete the data synchronization between nodes.

[0094] The above technical solution uses data volume threshold determination and the coordinated operation of stream load protocol encapsulation and targeted transmission process to filter large batches of synchronized data based on data volume threshold, generate stream load protocol HTTP push synchronization requests, send them to the target data nodes and complete the data import operation, thereby ensuring the consistency and timeliness of data in the distributed database cluster.

[0095] This invention provides a technical solution for standardized batch data import from the database driver's perspective. By acquiring HTTP push messages from the database driver, parsing these messages to obtain transaction identifiers and batch data, and determining the transaction processing batch based on the transaction identifiers, the solution performs an import operation on the batch data within that batch. It generates and returns a batch data import response result for the corresponding batch, which the database driver then uses to generate a communication connection closure request and sends it to the database server, closing the HTTP service process bound to the driver instance corresponding to the database driver. This solution achieves standardized batch data import processing from the database server's perspective. On one hand, by accurately parsing the transaction identifiers in the messages, it quickly locates the transaction processing batch, ensuring accurate matching between batch data and the target transaction. On the other hand, by promptly closing the corresponding HTTP service process after data import, it effectively releases system resources, preventing long-term idle resource occupation by service processes. Furthermore, the standardized message parsing and import process reduces the probability of errors in server-side data processing, ensuring the stability and reliability of data import and meeting the high-efficiency and secure requirements of various business applications for batch data reception and processing.

[0096] Example 3

[0097] Figure 3A This is an interaction diagram of the database server and database driver for a batch data import method based on the stream load protocol provided in Embodiment 3 of the present invention. This embodiment provides a preferred example based on the above embodiments.

[0098] S301. The database driver initiates a communication connection establishment request to the database server, requesting to establish a dedicated data transmission link with the server.

[0099] S302. After the database server verifies the legitimacy of the request, it establishes a communication connection with the driver. At the same time, it starts the driver instance that is uniquely corresponding to the driver and the HTTP service process bound to the instance, generates HTTP connection parameters, and feeds them back to the driver.

[0100] S303. After receiving the connection success message and storing the HTTP connection parameters, the database driver sends a transaction identifier request to the server.

[0101] S304. After receiving the request, the database server assigns a unique transaction identifier and records the association relationship, and then feeds the identifier back to the driver.

[0102] S305. The database driver responds to the user's batch data import request, extracts the batch data to be imported and counts the data volume; when the data volume reaches a preset threshold, it combines the stored HTTP connection parameters and the transaction identifier assigned by the server, encapsulates it into an HTTP push message according to the stream load protocol, and sends it to the server.

[0103] S306. The database server receives the HTTP push message, parses out the transaction identifier and batch data, matches the corresponding transaction processing batch by the identifier, executes the data writing operation in the database under the batch, generates a response result after completion, and sends it back to the driver.

[0104] S307. The database driver receives the response and sends a communication connection closure request to the server.

[0105] S308. After receiving the close request, the database server closes the driver instance and the bound HTTP service process corresponding to the driver, releases resources such as port and token, and sends a confirmation message of successful connection closure back to the driver.

[0106] Figure 3B This is an interaction diagram of the database server and database driver for another batch data import method based on the stream load protocol provided in Embodiment 3 of the present invention.

[0107] S411, The database driver sends a communication connection establishment request to the database server.

[0108] S412. The database server receives and verifies the request. If successful, it establishes a communication connection with the driver, generates a connection establishment success message, and sends it back.

[0109] S413. The database driver responds to the user's batch data import request, extracts the data to be imported and counts the data volume, and sends a data import start message to the server.

[0110] S414. After receiving the data import start message, the database server starts the HTTP service process, determines the HTTP connection parameters including the request port and authentication information, and generates a preparation completion message to be sent back to the driver.

[0111] S415. The database driver receives the preparation completion message and stores the connection parameters. Combined with the preset identifier negotiated with the server in advance, it generates an HTTP push message based on the stream load protocol (the request header contains a label field carrying the identifier) ​​and sends it to the server.

[0112] S416. The database server receives an HTTP push message, parses out the preset identifier and batch data, identifies the data source through the identifier, matches the corresponding transaction processing batch and executes the data import operation, generates a batch data import response result and sends it back.

[0113] S417. The database driver receives the import response result and confirms that the data import is complete, then sends a batch push completion message to the server.

[0114] S418. After receiving the batch push completion message, the database server closes the corresponding HTTP service process and generates a completion response message to send back to the driver.

[0115] Figure 3C This is an interaction diagram of the database server and database driver for another batch data import method based on the stream load protocol provided in Embodiment 3 of the present invention.

[0116] S511. When the database server starts, the HTTP service is started synchronously, and fixed HTTP connection-related parameters are configured. The database driver sends a communication connection establishment request to the server, which carries the preset HTTP connection parameter configuration information.

[0117] S512: The database server receives and verifies the request. Once the request is successful, it establishes a communication connection with the driver, generates a connection success message, and sends it back.

[0118] S513: The database driver responds to the user's batch data import request, extracts the data to be imported and counts the data volume, and generates an HTTP push message based on the stream load protocol in combination with the preset HTTP connection parameters, and sends the HTTP push message to the server.

[0119] S514. After receiving the message, the database server parses out the batch data, generates a transaction identifier by default, determines the transaction processing batch, executes the data import operation, generates the batch data import response result, and sends it back.

[0120] S515: The database driver receives the response result. If the import is successful, it sends a connection closure request to the server.

[0121] S516. After receiving the close request, the database server terminates the communication connection with the driver, generates a connection close confirmation message, and sends it back.

[0122] Example 4

[0123] Figure 4This is a schematic diagram of a batch data import device based on the stream load protocol, which can be deployed on a database driver, according to Embodiment 4 of the present invention. The batch data import device based on the stream load protocol, which can be deployed on a database driver, is suitable for scenarios involving the import of large amounts of data. This batch data import device based on the stream load protocol can be implemented in hardware and / or software. It can be applied to a batch data import method based on the stream load protocol and can be specifically configured in a controller, such as... Figure 4 As shown, the device includes: a data import module 401, an HTTP push message generation module 402, a response result generation module 403, and a close request generation module 404. Wherein:

[0124] The data import module 401 is used to respond to a batch data import request and determine the batch data and the amount of data to be imported based on the batch data import request.

[0125] The HTTP push message generation module 402 is used to determine if the amount of imported data reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server. Then, based on the pre-stored Hypertext Transfer Protocol (HTTP) connection-related parameters, transaction identifier, and batch data, it generates an HTTP push message using the stream load protocol.

[0126] The response result generation module 403 is used to send the HTTP push message back to the database server so that the database server can parse the HTTP push message to obtain the transaction identifier and batch data, determine the transaction processing batch based on the transaction identifier, perform import operation on the batch data under the transaction processing batch, and generate and send back the batch data import response result under the corresponding transaction processing batch.

[0127] The closure request generation module 404 is used to generate a communication connection closure request based on the batch data import response results, and send the communication connection closure request to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver.

[0128] The above-described embodiments of the invention can be applied to a database driver. In response to a batch data import request, the batch data and the amount of data to be imported are determined based on the request. If the amount of data to be imported reaches a preset data volume threshold and a transaction identifier previously obtained from the database server exists, an HTTP push message is generated based on the pre-stored HTTP connection parameters, the transaction identifier, and the batch data, using the stream load protocol. After the database server successfully establishes a communication connection with the database driver, it starts the HTTP service process bound to the driver instance corresponding to the database driver, and sends the HTTP push message back to the database server. The database server then parses the HTTP push message to obtain the transaction identifier and the batch data. Based on the transaction identifier, it determines a transaction processing batch and performs an import operation on the batch data within that batch. It generates and sends back the batch data import response result for the corresponding batch. Based on the batch data import response result, it generates a communication connection closure request and sends it to the database server, allowing the database server to close the HTTP service process bound to the driver instance corresponding to the database driver. The above technical solutions can significantly improve the efficiency and reliability of batch data import in databases. On the one hand, by reducing the number of data transmission interactions through the stream load protocol, large amounts of data can be imported quickly. On the other hand, the atomicity of data import is guaranteed by transaction identifiers, avoiding data loss or duplicate import issues and ensuring database data consistency. At the same time, the standardized parameter interaction between the driver and the server and the precise control of the HTTP service process reduce the risk of connection failure, optimize system resource usage, and meet the core needs of various businesses for high efficiency and high security in batch data import in databases.

[0129] Optionally, the device further includes:

[0130] The connection establishment success message feedback module is used to initiate a communication connection establishment request to the database server before responding to the batch data import request and determining the batch data and the amount of data to be imported based on the batch data import request. This allows the database server to establish a communication connection based on the communication connection establishment request. At the same time, it starts the HTTP service process bound to the driver instance corresponding to the database driver. After the communication connection is successfully established, it determines the HTTP connection-related parameters and generates and sends back a connection establishment success message based on the HTTP connection-related parameters.

[0131] Optionally, the device further includes:

[0132] The HTTP push message generation module is used to determine if the imported data volume reaches a preset data volume threshold and there is no transaction identifier obtained from the database server in advance. Then, based on the pre-stored HTTP connection-related parameters and the preset identifier, it generates an HTTP push message according to the stream load protocol.

[0133] The HTTP push message request header's tag field carries a preset identifier; the tag field is a field defined in the streamload protocol.

[0134] The response result generation module is used to send HTTP push messages back to the database server so that the database server can parse the HTTP push messages to obtain preset identifiers and batch data. If it is determined from the preset identifier that there is a transaction commit request initiated by the database driver within a preset time period, the transaction processing batch is determined according to the transaction commit request, and the batch data is imported under the transaction processing batch. The module generates and sends back the batch data import response result under the corresponding transaction processing batch.

[0135] The batch data import device provided in this embodiment of the invention can execute a batch data import method based on the stream load protocol provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0136] Example 5

[0137] Figure 5 This is a schematic diagram of a batch data import device based on the stream load protocol, which can be deployed on a database server, according to Embodiment 5 of the present invention. The batch data import device based on the stream load protocol, which can be deployed on a database server, is suitable for scenarios involving the import of large amounts of data. This batch data import device based on the stream load protocol can be implemented in hardware and / or software. It can be applied to a batch data import method based on the stream load protocol and can be specifically configured in a controller, such as... Figure 5 As shown, the device includes: an HTTP push message acquisition module 501, an HTTP push message parsing module 502, a batch data import response result generation module 503, an HTTP service process shutdown module 504, and an HTTP push message determination module 505. Wherein:

[0138] The HTTP push message acquisition module 501 is used to acquire HTTP push messages from the database driver.

[0139] The HTTP push message parsing module 502 is used to parse HTTP push messages to obtain transaction identifiers and batch data.

[0140] The batch data import response result generation module 503 is used to determine the transaction processing batch according to the transaction identifier, perform batch data import operation under the transaction processing batch, generate and return the batch data import response result under the corresponding transaction processing batch, so that the database driver can generate a communication connection closure request based on the batch data import response result and send the communication connection closure request to the database server.

[0141] The HTTP service process shutdown module 504 is used to shut down the HTTP service process bound to the driver instance corresponding to the database driver.

[0142] The HTTP push message determination module 505 is used by the database driver to respond to a batch data import request, determine the batch data and the amount of data to be imported according to the batch data import request, and when it is determined that the amount of data to be imported reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server, generate an HTTP push message based on the stream load protocol according to the pre-stored Hypertext Transfer Protocol (HTTP) connection-related parameters, the transaction identifier and the batch data.

[0143] The above-described embodiments of the invention can be applied to a database server. By acquiring HTTP push messages from the database driver, the HTTP push messages are parsed to obtain transaction identifiers and batch data. Based on the transaction identifiers, a transaction processing batch is determined, and the batch data is imported within that batch. A batch data import response result is generated and fed back, allowing the database driver to generate a communication connection closure request based on the batch data import response result and send the request to the database server, thus closing the HTTP service process bound to the driver instance corresponding to the database driver. This technical solution achieves standardized processing of batch data import from the database server. On one hand, by accurately parsing the transaction identifiers in the messages, the transaction processing batch is quickly located, ensuring accurate matching between batch data and the target transaction. On the other hand, by promptly closing the corresponding HTTP service process after data import, system resources are effectively released, preventing the service process from idling and occupying resources for extended periods. Simultaneously, the standardized message parsing and import process reduces the probability of errors in server-side data processing, ensuring the stability and reliability of data import and meeting the high-efficiency and secure requirements of various businesses for batch data reception and processing in the database.

[0144] Optionally, the device further includes:

[0145] The HTTP push synchronization request generation module is used to respond to data synchronization requests, determine the amount of data to be synchronized, and if the amount of data to be synchronized reaches a preset data amount threshold, then generate an HTTP push synchronization request based on the stream load protocol according to the data to be synchronized.

[0146] The data import module is used to determine the target data node based on the data synchronization request, and send the HTTP push synchronization request to the target data node so that the target data node can determine the data to be synchronized based on the HTTP push synchronization request and perform the data import operation on the data to be synchronized.

[0147] The batch data import device provided in this embodiment of the invention can execute a batch data import method based on the stream load protocol provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0148] Example 6

[0149] Figure 6 A schematic diagram of an electronic device 60 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0150] like Figure 6 As shown, the electronic device 60 includes at least one processor 61 and a memory, such as a read-only memory (ROM) 62 and a random access memory (RAM) 63, communicatively connected to the at least one processor 61. The memory stores computer programs executable by the at least one processor. The processor 61 can perform various appropriate actions and processes based on the computer program stored in the ROM 62 or loaded from storage unit 68 into the RAM 63. The RAM 63 can also store various programs and data required for the operation of the electronic device 60. The processor 61, ROM 62, and RAM 63 are interconnected via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0151] Multiple components in electronic device 60 are connected to I / O interface 65, including: input unit 66, such as keyboard, mouse, etc.; output unit 67, such as various types of monitors, speakers, etc.; storage unit 68, such as disk, optical disk, etc.; and communication unit 69, such as network card, modem, wireless transceiver, etc. Communication unit 69 allows electronic device 60 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0152] Processor 61 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 61 performs the various methods and processes described above, such as a batch data import method based on a stream load protocol.

[0153] In some embodiments, a batch data import method based on a stream load protocol may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 60 via ROM 62 and / or communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the batch data import method based on a stream load protocol described above may be performed. Alternatively, in other embodiments, processor 61 may be configured as a batch data import method based on a stream load protocol by any other suitable means (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0158] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0159] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0160] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0161] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A batch data import method based on the stream load protocol, characterized in that, Applied to the database driver side, including: In response to a batch data import request, determine the batch data and the amount of data to be imported based on the batch data import request; If the imported data volume reaches a preset data volume threshold and a transaction identifier obtained in advance from the database server exists, then an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, the transaction identifier, and the batch data. The HTTP connection-related parameters are generated and fed back by the database server after the database driver successfully establishes a communication connection with the database server. After the database server successfully establishes a communication connection with the database driver, it starts the HTTP service process bound to the driver instance corresponding to the database driver. The HTTP push message is sent back to the database server so that the database server can parse the HTTP push message to obtain the transaction identifier and batch data. The transaction processing batch is determined according to the transaction identifier, and the batch data is imported under the transaction processing batch. The corresponding batch data import response result is generated and fed back. Based on the batch data import response result, a communication connection closure request is generated and sent to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver.

2. The method according to claim 1, characterized in that, Before determining the batch data and the amount of data to be imported based on the batch data import request in response to the batch data import request, the method further includes: A communication connection establishment request is initiated to the database server so that the database server can establish a communication connection according to the communication connection establishment request. At the same time, the HTTP service process bound to the driver instance corresponding to the database driver is started. After the communication connection is successfully established, the HTTP connection-related parameters are determined, and a connection establishment success message is generated and fed back based on the HTTP connection-related parameters.

3. The method according to claim 1, characterized in that, The method further includes: If the amount of imported data reaches a preset data volume threshold and there is no transaction identifier obtained from the database server in advance, then an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters and the preset identifier; the preset identifier is carried in the tag field of the request header of the HTTP push message; the tag field is a field defined in the stream load protocol. The HTTP push message is fed back to the database server so that the database server can parse the HTTP push message to obtain a preset identifier and batch data. If it is determined from the preset identifier that there is a transaction commit request initiated by the database driver within a preset time period, then the transaction processing batch is determined from the transaction commit request, and the batch data is imported under the transaction processing batch, generating and feeding back the batch data import response result under the corresponding transaction processing batch.

4. A batch data import method based on the stream load protocol, characterized in that, Applied to the database server, including: Retrieve HTTP push messages from the database driver; The HTTP push message is parsed to obtain the transaction identifier and batch data; Based on the transaction identifier, the transaction batch is determined, and the batch data is imported under the transaction batch. The batch data import response result under the corresponding transaction batch is generated and fed back, so that the database driver can generate a communication connection closure request based on the batch data import response result and send the communication connection closure request to the database server. Shut down the HTTP service process bound to the driver instance corresponding to the database driver; The HTTP push message is generated in the following way: In response to a batch data import request, the database driver determines the batch data and the amount of data to be imported based on the batch data import request. When it is determined that the amount of data to be imported reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server, an HTTP push message is generated based on the stream load protocol according to the pre-stored HTTP connection-related parameters, the transaction identifier, and the batch data. The HTTP connection-related parameters are generated and fed back by the database server after the database driver and the database server successfully establish a communication connection; after the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started.

5. The method according to claim 4, characterized in that, The method further includes: In response to a data synchronization request, the amount of data to be synchronized is determined. If the amount of data to be synchronized reaches a preset data amount threshold, an HTTP push synchronization request is generated based on the stream load protocol according to the data to be synchronized. Based on the data synchronization request, a target data node is determined, and the HTTP push synchronization request is sent to the target data node so that the target data node can determine the data to be synchronized based on the HTTP push synchronization request and perform data import operations on the data to be synchronized.

6. A batch data import device based on the stream load protocol, characterized in that, Configured on the database driver side, including: The data import module is used to respond to a batch data import request and determine the batch data and the amount of data to be imported based on the batch data import request. The HTTP push message generation module is used to generate an HTTP push message based on the stream load protocol, according to pre-stored HTTP connection-related parameters, the transaction identifier, and batch data, if the imported data volume reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server; the HTTP connection-related parameters are generated and fed back by the database server after the database driver successfully establishes a communication connection with the database server; after the database server successfully establishes a communication connection with the database driver, it starts the HTTP service process bound to the driver instance corresponding to the database driver. The response result generation module is used to send the HTTP push message back to the database server, so that the database server can parse the HTTP push message to obtain the transaction identifier and batch data, determine the transaction processing batch according to the transaction identifier, and perform import operation on the batch data under the transaction processing batch, and generate and send back the batch data import response result under the corresponding transaction processing batch. The request generation module is used to generate a communication connection closure request based on the batch data import response results, and send the communication connection closure request to the database server so that the database server can close the HTTP service process bound to the driver instance corresponding to the database driver.

7. A batch data import device based on the stream load protocol, characterized in that, Configured on the database server, including: The HTTP push message acquisition module is used to acquire HTTP push messages from the database driver. The HTTP push message parsing module is used to parse the HTTP push message to obtain the transaction identifier and batch data; The batch data import response result generation module is used to determine the transaction processing batch based on the transaction identifier, perform the batch data import operation under the transaction processing batch, generate and return the batch data import response result under the corresponding transaction processing batch, so that the database driver can generate a communication connection closure request based on the batch data import response result and send the communication connection closure request to the database server. The HTTP service process shutdown module is used to shut down the HTTP service process bound to the driver instance corresponding to the database driver. The HTTP push message determination module is used by the database driver to respond to a batch data import request, determine the batch data and the amount of data to be imported based on the batch data import request, and when it is determined that the amount of data to be imported reaches a preset data volume threshold and there is a transaction identifier obtained in advance from the database server, generate an HTTP push message based on the stream load protocol according to the pre-stored Hypertext Transfer Protocol (HTTP) connection-related parameters, the transaction identifier and the batch data. The HTTP connection-related parameters are generated and fed back by the database server after the database driver and the database server successfully establish a communication connection; after the database server and the database driver successfully establish a communication connection, the HTTP service process bound to the driver instance corresponding to the database driver is started.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a batch data import method based on the stream load protocol according to any one of claims 1-3 and / or 4-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement a batch data import method based on the stream load protocol according to any one of claims 1-3 and / or 4-5.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a batch data import method based on the stream load protocol according to any one of claims 1-3 and / or 4-5.

Images