Data transmission method and device, computer device, readable storage medium and program product

Abstract

The application relates to a data transmission method and device, computer equipment, a computer readable storage medium and a computer program product. The method is applied to a network service system and comprises the following steps: in response to receiving a data transmission instruction, determining a target data volume corresponding to the data transmission instruction; acquiring a communication data file corresponding to the network service data according to the target data volume, and generating an initial transmission file matched with the communication data file, wherein the data volume of the communication data file is the target data volume; periodically acquiring the initial transmission file according to a preset period, and sending the initial transmission file to a target platform; and deleting the initial transmission file in the case of completed sending. The method can improve safety and save resources.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data transmission method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the booming development of the internet, more and more companies are choosing to deploy their services and products on networks for users. To ensure the security and reliability of these products, network packet capture can be used for monitoring. Network packet capture is a network diagnostic and analysis technique that captures and analyzes network data packets to understand the transmission of data packets within the network, enabling the assessment of network performance in response to network faults.

[0003] In related technologies, packet capture files can be obtained through the network packet capture module in the system and transmitted to the monitoring platform. However, this method requires significant storage resources, makes it difficult to perform long-term packet capture, and cannot guarantee the security and reliability of data services. Summary of the Invention

[0004] Therefore, it is necessary to provide a data transmission method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve security and save resources in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a data transmission method, which is applied to a network service system and includes:

[0006] In response to receiving a data transmission instruction, the target data volume corresponding to the data transmission instruction is determined;

[0007] According to the target data volume, obtain the communication data file corresponding to the network service data, and generate an initial transmission file that matches the communication data file, wherein the data volume of the communication data file is the target data volume;

[0008] According to a preset cycle, the initial transmission file is periodically acquired and sent to the target platform. Once the transmission is complete, the initial transmission file is deleted.

[0009] In one embodiment, the periodic acquisition of the initial transfer file and its transmission to the target platform includes:

[0010] Periodically acquire multiple initial transmission files corresponding to the current period, wherein the initial transmission files include data acquisition parameters and communication data;

[0011] Delete the data acquisition parameters from all files in the initial transmission file except the target file to obtain the processed initial transmission file;

[0012] The processed initial transfer file is sent to the target platform.

[0013] In one embodiment, the method for determining the target file includes:

[0014] Determine the file generation time corresponding to each of the initial transmission files;

[0015] From the multiple initial transmission files, the initial transmission file with the earliest file generation time is determined as the target file.

[0016] In one embodiment, the step of determining the target data volume corresponding to the data transmission instruction in response to receiving the data transmission instruction includes:

[0017] In response to receiving a data transmission instruction, the resource utilization rate of the network service system is determined;

[0018] If the resource occupancy rate is less than or equal to a preset threshold, the target data volume corresponding to the data transmission instruction is determined.

[0019] In one embodiment, determining the resource utilization rate corresponding to the network service system includes:

[0020] The target resource type is determined based on the data transmission instructions;

[0021] Determine the initial resource occupancy rate corresponding to the target resource type in the network service system, and generate a resource occupancy rate based on the initial resource occupancy rate.

[0022] In one embodiment, the method further includes:

[0023] When the resource occupancy rate of the network service system exceeds a preset threshold, the acquisition of communication data files is stopped.

[0024] Secondly, this application also provides a data transmission apparatus, which is applied to a network service system and includes:

[0025] The receiving module is used to determine the target data volume corresponding to the received data transmission instruction in response to receiving the data transmission instruction;

[0026] The acquisition module is used to acquire the communication data file corresponding to the network service data according to the target data volume, and generate an initial transmission file that matches the communication data file, wherein the data volume of the communication data file is the target data volume;

[0027] The sending module is used to periodically acquire the initial transmission file according to a preset period, send it to the target platform, and delete the initial transmission file after the sending is completed.

[0028] In one embodiment, the sending module is further configured to:

[0029] Periodically acquire multiple initial transmission files corresponding to the current period, wherein the initial transmission files include data acquisition parameters and communication data;

[0030] Delete the data acquisition parameters from all files in the initial transmission file except the target file to obtain the processed initial transmission file;

[0031] The processed initial transfer file is sent to the target platform.

[0032] In one embodiment, the apparatus further includes a target file determination module, the target file determination module being used to:

[0033] Determine the file generation time corresponding to each of the initial transmission files;

[0034] From the multiple initial transmission files, the initial transmission file with the earliest file generation time is determined as the target file.

[0035] In one embodiment, the receiving module is further configured to:

[0036] In response to receiving a data transmission instruction, the resource utilization rate of the network service system is determined;

[0037] If the resource occupancy rate is less than or equal to a preset threshold, the target data volume corresponding to the data transmission instruction is determined.

[0038] In one embodiment, the receiving module is further configured to:

[0039] The target resource type is determined based on the data transmission instructions;

[0040] Determine the initial resource occupancy rate corresponding to the target resource type in the network service system, and generate a resource occupancy rate based on the initial resource occupancy rate.

[0041] In one embodiment, the device is further configured to:

[0042] When the resource occupancy rate of the network service system exceeds a preset threshold, the acquisition of communication data files is stopped.

[0043] Thirdly, embodiments of this disclosure also provide a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the embodiments of this disclosure.

[0044] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in any one of the embodiments of this disclosure.

[0045] Fifthly, embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the embodiments of this disclosure.

[0046] The aforementioned data transmission method, apparatus, computer equipment, computer-readable storage medium, and computer program product are used to capture relevant data during the operation of a network service system. Upon receiving a data transmission instruction, the corresponding target data volume is determined, the communication data file corresponding to the data transmission instruction is obtained according to the target data volume, and an initial transmission file matching the communication data file is generated. Simultaneously, the initial transmission file is periodically acquired and sent to the target platform according to a preset cycle. After sending, the initial transmission file is deleted. This enables data capture and uploading during the operation of the network service system, ensuring the security and reliability of the network service system. During data capture, the communication data file is obtained according to the target data volume, rather than the entire file package. Therefore, when uploading to the target platform, the initial transmission file can be periodically acquired and uploaded directly without waiting for the entire file package to be acquired, ensuring data upload efficiency. Furthermore, deleting the initial transmission file directly after uploading effectively reduces the storage space occupied by the transmission file, saving resources. This achieves data fragmentation for acquisition, generation, sending, and deletion, ensuring data transmission efficiency and stability, and making it suitable for more application scenarios. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating a data transmission method in one embodiment;

[0049] Figure 2 This is a schematic diagram of the initial file transfer in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the method for determining the target file in one embodiment;

[0051] Figure 4This is a flowchart illustrating the data transmission method in another embodiment;

[0052] Figure 5 This is a structural block diagram of a data transmission device in one embodiment;

[0053] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1 As shown, a data transmission method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method is applied to a network service system, including:

[0056] Step S110: In response to receiving a data transmission instruction, determine the target data volume corresponding to the data transmission instruction;

[0057] For example, a network service system can be a service system deployed on a network to provide network services to relevant users. During the interaction with users, communication data files are generated. Through the communication data files, the data transmission status of the network service system can be understood, network faults can be diagnosed, and network performance can be evaluated.

[0058] Optionally, the data transmission command can be sent when data monitoring begins, periodically, or to the operations and maintenance account, depending on the specific application scenario.

[0059] In one example, the target data volume can be the file slice size determined based on the actual application scenario. In another example, preset parameters, such as the parameter "-C", can be used to split the packet capture file (i.e., the original communication data file) into multiple smaller files.

[0060] Step S120: According to the target data volume, obtain the communication data file corresponding to the network service data, and generate an initial transmission file that matches the communication data file, wherein the data volume of the communication data file is the target data volume;

[0061] Typically, when capturing communication data files, the resulting packet capture files are quite large. To ensure data transmission efficiency, this embodiment sets a target data volume. When capturing data, the communication data file is captured according to the target data volume. For example, the original communication data file can be split into multiple smaller files, i.e., communication data files, according to the target data volume.

[0062] In one example, communication data files can be continuously captured, and the actions of capturing, generating, sending, and deleting can be performed simultaneously.

[0063] Optionally, while capturing the communication data file, a corresponding initial transmission file is generated. In one example, the initial transmission file is generated in the corresponding target directory. The initial transmission file can be obtained by processing the communication data file according to the file format corresponding to the data packet capture scenario. For example, the initial book-breaking file may include a PCAP file.

[0064] Step S130: Periodically acquire the initial transmission file according to a preset period and send it to the target platform. After the transmission is completed, delete the initial transmission file.

[0065] For example, the preset period can be a data transmission period determined based on the actual application scenario. Initial transmission files are periodically acquired and sent to the target platform. In one example, the target platform may include, but is not limited to, the cloud.

[0066] Optionally, after the initial transfer file is sent, it can be deleted to reduce the use of storage resources.

[0067] This embodiment of the disclosure is used to capture relevant data during the operation of a network service system. Upon receiving a data transmission instruction, the corresponding target data volume is determined, and the communication data file corresponding to the data transmission instruction is obtained according to the target data volume. An initial transmission file matching the communication data file is generated. Simultaneously, the initial transmission file is periodically obtained and sent to the target platform according to a preset cycle. After sending is completed, the initial transmission file is deleted. This enables data capture and uploading during the operation of the network service system, ensuring the security and reliability of the network service system. During data capture, the communication data file is obtained according to the target data volume, rather than the entire file package. This allows for the periodic acquisition and uploading of the initial transmission file directly to the target platform without waiting for the entire file package to be acquired, ensuring data upload efficiency. Furthermore, deleting the initial transmission file directly after uploading effectively reduces the storage space occupied by the transmission file, saving resources. This achieves data fragmentation for acquisition, generation, sending, and deletion, ensuring data transmission efficiency and stability, and is applicable to more application scenarios.

[0068] In one embodiment, the periodic acquisition of the initial transfer file and its transmission to the target platform includes:

[0069] Periodically acquire multiple initial transmission files corresponding to the current period, wherein the initial transmission files include data acquisition parameters and communication data;

[0070] Delete the data acquisition parameters from all files in the initial transmission file except the target file to obtain the processed initial transmission file;

[0071] The processed initial transfer file is sent to the target platform.

[0072] For example, the initial transmission file may include data acquisition parameters and communication data. The data acquisition parameters may include, but are not limited to, the file modulus, version number, timestamp precision, data link type, etc. The same data transmission instruction corresponds to the same data acquisition parameters.

[0073] Figure 2 This is a schematic diagram illustrating an initial file transfer according to an exemplary embodiment, with reference to... Figure 2 As shown, taking the initial transmission file as a PCAP file as an example, the structure of a PCAP file can be divided into two main parts: the global file header (i.e., data acquisition parameters) and the data packet block (i.e., communication data). The global file header is located at the beginning of the PCAP file, generally occupying 24 bytes, and contains some key information, such as the file modulus, version number, timestamp precision, data link type, etc.

[0074] Optionally, if multiple initial transmission files contain the same data acquisition parameters, then the data acquisition parameters in all files except the target file are deleted to obtain the processed initial transmission file. The target file can be determined based on the actual application scenario; for example, the target file can be the first file among multiple initial transmission files. In one example, since they have the same command parameters (i.e., data transmission instructions), only the global header of the first PCAP file needs to be considered. If it is necessary to concatenate them into a large PCAP file, the header of the first PCAP file is retained, and the first 24 bytes of subsequent PCAP files are truncated before concatenating the data packets. In the Linux shell environment, the `cut` command can be used to directly truncate the file.

[0075] The processed transmission file is sent to the target platform. In one example, the target file can be the first file among multiple initial transmission files. When sending to the target platform, the target file is sent first, and the other deleted files are sent in sequence. The file received by the target platform is a concatenation of the data acquisition parameters of the target file and the communication data of multiple initial transmission files.

[0076] In this embodiment of the disclosure, when the initial transmission file is acquired and uploaded, based on the structure of the initial transmission file, data acquisition parameters in some files are deleted during transmission, which effectively reduces the workload during data transmission and improves data transmission efficiency. Furthermore, by setting the target file, the integrity and reliability of the data are guaranteed.

[0077] In one embodiment, such as Figure 3 As shown, the method for determining the target file includes:

[0078] Step S310: Determine the file generation time corresponding to each of the initial transmission files;

[0079] Step S320: From the plurality of initial transmission files, determine the initial transmission file with the earliest file generation time as the target file.

[0080] For example, the target file is determined based on the file generation time. In one example, the initial transfer files can be generated sequentially, corresponding to file generation times.

[0081] In one example, the initial file whose file creation time is earliest is determined as the target file. Optionally, when transferring files to the target platform, they are transferred sequentially according to their file creation time.

[0082] In this embodiment of the disclosure, the target file is determined based on the file generation time of the initial transmission file, thereby enabling the target file to be located quickly and accurately from multiple initial transmission files, and then subsequent data interception and transmission are carried out to ensure the real-time performance and accuracy of data transmission.

[0083] In one embodiment, determining the target data volume corresponding to the data transmission instruction in response to receiving the data transmission instruction includes:

[0084] In response to receiving a data transmission instruction, the resource utilization rate of the network service system is determined;

[0085] If the resource occupancy rate is less than or equal to a preset threshold, the target data volume corresponding to the data transmission instruction is determined.

[0086] For example, in response to receiving a data transmission instruction, the resource utilization rate of the network service system is determined. In one example, the resource utilization rate can be the utilization rate of the computing resources of the processor corresponding to the network service system, which can be determined according to the actual application scenario.

[0087] When the resource utilization rate is less than or equal to a preset threshold, it can be assumed that the network service system has sufficient remaining resources, allowing for packet capture. Therefore, the subsequent process continues. The preset threshold can be determined based on the actual application scenario. In one example, the preset threshold can be sent via a data transmission command.

[0088] In this embodiment of the disclosure, upon receiving a data transmission instruction, the resource occupancy rate of the network service system is first determined. If the resource occupancy rate is low, subsequent data acquisition and transmission steps are executed. This enables the data capture process to be initiated based on the resource occupancy rate of the network service system in the actual application scenario, ensuring the stability of the network service system operation and avoiding problems such as network service system lag caused by data capture. This makes it suitable for more application scenarios.

[0089] In one embodiment, determining the resource utilization rate corresponding to the network service system includes:

[0090] The target resource type is determined based on the data transmission instructions;

[0091] Determine the initial resource occupancy rate corresponding to the target resource type in the network service system, and generate a resource occupancy rate based on the initial resource occupancy rate.

[0092] For example, when determining resource utilization, the target resource type is first determined based on the data transmission instructions. The target resource type can be determined from the resource types corresponding to the network service system. The resource type may include, but is not limited to, processor computing resources, memory resources, etc., and can be specifically determined based on the actual application scenario.

[0093] Optionally, the initial resource utilization rate corresponding to the target resource type in the network service system is determined. Different target resource types may correspond to different initial resource utilization rates, and when there are multiple target resource types, there may be multiple initial resource utilization rates.

[0094] In one example, resource utilization is generated based on initial resource utilization. For instance, the sum of initial resource utilization can be used to determine resource utilization, or a weighted sum can be calculated based on preset weights to obtain resource utilization.

[0095] According to the embodiments disclosed herein, a specified target resource type can be determined and the resource utilization rate can be calculated based on the data transmission instructions. This enables flexible calculation of the resource utilization rate, and the target resource type can be specified according to actual needs. Compared with data capture from components and built-in modules, this method is more flexible and applicable to more application scenarios.

[0096] In one embodiment, the method further includes:

[0097] When the resource occupancy rate of the network service system exceeds a preset threshold, the acquisition of communication data files is stopped.

[0098] For example, the resource utilization rate of the network service system is monitored in real time, and when the resource utilization rate of the network service system exceeds a preset threshold, the acquisition of communication data files is stopped.

[0099] In one example, if resource utilization exceeds a preset threshold, it can be assumed that the network service system is overloaded, and continuing data scraping may lead to service lag or other issues. The preset threshold can be set based on the actual application scenario. In another example, both generating and sending the initial transfer file can be stopped simultaneously.

[0100] Optionally, when the resource utilization rate is less than or equal to a preset threshold, communication data files can continue to be acquired and subsequently generated and transmitted.

[0101] In this embodiment of the disclosure, the resource utilization rate of the network service system is monitored during data transmission. If the resource utilization rate exceeds a preset threshold, the acquisition of communication data files is stopped, i.e., the data capture process is stopped. This allows for automatic cessation of data capture based on resource utilization, improving the flexibility of data transmission and making it suitable for more application scenarios.

[0102] Figure 4 This is a flowchart illustrating a data transmission method according to an exemplary embodiment, with reference to... Figure 4As shown, you can first set up an object storage, ordinary cloud disk, or compliant log monitoring platform in the network service system to support file chunked upload and storage services. In one example, data capture can be performed using data transfer software, such as Tcpdump. Install the data transfer software on the target machine corresponding to the network service system. Execute the following steps using the preset program: a) Monitor the machine's resource utilization and control Tcpdump to start and stop packet capture (data capture) based on the resource utilization. When capturing packets, use the Tcpdump parameter "-C" to split the captured file into multiple smaller files. b) After executing the Tcpdump command for a period of time, multiple files "output.pcap, output.pcap1, output.pcap2..." (i.e., initial transmission files) will be generated in the target directory. All captured files need special handling as a file stream. During transmission, retain the header of the first PCAP file, and truncate the first 24 bytes of subsequent PCAP files, then concatenate the data packets. In the Linux shell environment, the cut command can be used to directly truncate the files. c) Upload the processed PCAP file from the previous step to the cloud. In a Linux shell environment, the curl command can be used to POST and upload file chunks. d) Delete the file immediately after upload to reduce the consumption of machine storage resources.

[0103] This embodiment provides a solution for dynamically detecting system resource occupancy and automatically starting and stopping packet capture. It can easily and loosely achieve long-term low-resource-occupancy network packet capture. Data transmission is performed through software programs and scripts, which can be extended to meet more complex needs and is applicable to more application scenarios.

[0104] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0105] Based on the same inventive concept, this application also provides a data transmission apparatus for implementing the data transmission method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, specific limitations in one or more data transmission apparatus embodiments provided below can be found in the limitations of the data transmission method described above, and will not be repeated here.

[0106] In one exemplary embodiment, such as Figure 5 As shown, a data transmission device 500 is provided, which is applied to a network service system and includes:

[0107] The receiving module 510 is configured to determine the target data volume corresponding to the data transmission instruction in response to receiving the data transmission instruction;

[0108] The acquisition module 520 is used to acquire the communication data file corresponding to the network service data according to the target data volume, and generate an initial transmission file that matches the communication data file, wherein the data volume of the communication data file is the target data volume;

[0109] The sending module 530 is used to periodically acquire the initial transmission file according to a preset period, send it to the target platform, and delete the initial transmission file after the sending is completed.

[0110] In one embodiment, the sending module is further configured to:

[0111] Periodically acquire multiple initial transmission files corresponding to the current period, wherein the initial transmission files include data acquisition parameters and communication data;

[0112] Delete the data acquisition parameters from all files in the initial transmission file except the target file to obtain the processed initial transmission file;

[0113] The processed initial transfer file is sent to the target platform.

[0114] In one embodiment, the apparatus further includes a target file determination module, the target file determination module being used to:

[0115] Determine the file generation time corresponding to each of the initial transmission files;

[0116] From the multiple initial transmission files, the initial transmission file with the earliest file generation time is determined as the target file.

[0117] In one embodiment, the receiving module is further configured to:

[0118] In response to receiving a data transmission instruction, the resource utilization rate of the network service system is determined;

[0119] If the resource occupancy rate is less than or equal to a preset threshold, the target data volume corresponding to the data transmission instruction is determined.

[0120] In one embodiment, the receiving module is further configured to:

[0121] The target resource type is determined based on the data transmission instructions;

[0122] Determine the initial resource occupancy rate corresponding to the target resource type in the network service system, and generate a resource occupancy rate based on the initial resource occupancy rate.

[0123] In one embodiment, the device is further configured to:

[0124] When the resource occupancy rate of the network service system exceeds a preset threshold, the acquisition of communication data files is stopped.

[0125] Each module in the aforementioned data transmission device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0126] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores data such as communication data files, as described in this embodiment. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data transmission method.

[0127] Those skilled in the art will understand that Figure 6The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0128] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0129] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0130] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0134] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data transmission method, characterized in that, The method is applied to a network service system, including: In response to receiving a data transmission instruction, the target data volume corresponding to the data transmission instruction is determined; According to the target data volume, obtain the communication data file corresponding to the network service data, and generate an initial transmission file that matches the communication data file, wherein the data volume of the communication data file is the target data volume; According to a preset cycle, the initial transmission file is periodically acquired and sent to the target platform. Once the transmission is complete, the initial transmission file is deleted. The process of periodically acquiring the initial transmission file and sending it to the target platform includes: periodically acquiring multiple initial transmission files corresponding to the current period, wherein the initial transmission files include data acquisition parameters and communication data; deleting the data acquisition parameters from other files in the initial transmission files except the target file to obtain a processed initial transmission file; and sending the processed initial transmission file to the target platform.

2. The method according to claim 1, characterized in that, The methods for determining the target file include: Determine the file generation time corresponding to each of the initial transmission files; From the multiple initial transmission files, the initial transmission file with the earliest file generation time is determined as the target file.

3. The method according to claim 1, characterized in that, The step of determining the target data volume corresponding to the data transmission instruction in response to receiving the data transmission instruction includes: In response to receiving a data transmission instruction, the resource utilization rate of the network service system is determined; If the resource occupancy rate is less than or equal to a preset threshold, the target data volume corresponding to the data transmission instruction is determined.

4. The method according to claim 1, characterized in that, Determining the resource utilization rate corresponding to the network service system includes: The target resource type is determined based on the data transmission instructions; Determine the initial resource occupancy rate corresponding to the target resource type in the network service system, and generate a resource occupancy rate based on the initial resource occupancy rate.

5. The method according to claim 1, characterized in that, The method further includes: When the resource occupancy rate of the network service system exceeds a preset threshold, the acquisition of communication data files is stopped.

6. A data transmission device, characterized in that, The device is applied to a network service system and includes: The receiving module is used to determine the target data volume corresponding to the received data transmission instruction in response to receiving the data transmission instruction; The acquisition module is used to acquire the communication data file corresponding to the network service data according to the target data volume, and generate an initial transmission file that matches the communication data file, wherein the data volume of the communication data file is the target data volume; The sending module is used to periodically acquire the initial transmission file according to a preset period and send it to the target platform. After the transmission is completed, the initial transmission file is deleted. The sending module is also used to: periodically acquire multiple initial transmission files corresponding to the current period, wherein the initial transmission files include data acquisition parameters and communication data; delete the data acquisition parameters from the initial transmission files other than the target file to obtain a processed initial transmission file; and send the processed initial transmission file to the target platform.

7. The apparatus according to claim 6, characterized in that, The apparatus further includes a target file determination module, the target file determination module being used for: Determine the file generation time corresponding to each of the initial transmission files; From the multiple initial transmission files, the initial transmission file with the earliest file generation time is determined as the target file.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images