Data transmission method and device, electronic equipment and storage medium

Abstract

The invention relates to a data transmission method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining to-be-transmitted data through a public information channel, the to-be-transmitted data comprising to-be-transmitted event data and to-be-transmitted file data; sending to-be-transmitted event data and to-be-transmitted file data meeting a preset transmission condition in the to-be-transmitted data to a data transmission queue; obtaining a mapping relationship between a file descriptor and a multiplexing descriptor of each piece of to-be-transmitted data recorded in the data transmission queue; and according to the mapping relationship, sequentially transmitting the target transmission data in the data transmission queue. The to-be-transmitted data is obtained through the public information channel, thread resources do not need to be occupied independently, and CPU thread resources are saved; efficient data processing and transmission are achieved through the mapping relation between the file descriptor and the multiplexing descriptor, consumption of system resources is reduced, and resource waste caused by thread blocking is avoided.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data transmission method, device, electronic device and storage medium. Background Art

[0002] Zero-copy technology has been widely used in low-latency network interface cards and multiple latency-sensitive fields. For network processing systems such as network interface cards, zero-copy technology means that after data is transferred from hardware to direct memory access (DMA) memory, no additional data copying is required. Even during the conversion process from memory state to application state, data does not need to be copied again, thus achieving zero copy and achieving extremely fast data processing speed.

[0003] However, the existing low-latency network interface card zero-copy technology requires each connection to occupy a CPU thread for polling processing to achieve the best latency performance; if concurrent processing of multiple connections is to be achieved, more CPU resources and other system resources will be required, which undoubtedly poses a huge challenge to the performance of the server.

[0004] Therefore, there is an urgent need to develop a data transmission method, device, electronic device and storage medium to solve one or more of the above-mentioned problems. Summary of the invention

[0005] In view of this, in order to solve the above technical problems or part of the technical problems, the embodiments of the present invention provide a data transmission method, device, electronic device and storage medium.

[0006] In a first aspect, the present application provides a data transmission method, the method comprising:

[0007] Acquire data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted;

[0008] Sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions among the data to be transmitted to the data transmission queue;

[0009] Obtaining a mapping relationship between a file descriptor of each data to be transmitted and a multiplexing descriptor recorded in the data transmission queue;

[0010] According to the mapping relationship, the target transmission data in the data transmission queue is transmitted sequentially.

[0011] In a possible implementation manner, sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission condition in the data to be transmitted to the data transmission queue includes:

[0012] Sending the event data to be transmitted included in the data to be transmitted to a queue of events to be transmitted;

[0013] Determine whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition, and obtain a determination result;

[0014] According to the judgment result, the queue of events to be transmitted is updated so that the event data recorded in the queue of events to be transmitted is the event data to be transmitted that meets the preset transmission condition;

[0015] The event data to be transmitted and the file data to be transmitted recorded in the event queue to be transmitted are sent to the data transmission queue.

[0016] In a possible implementation manner, the determining whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition to obtain a determination result includes:

[0017] Obtaining the file descriptor of the event data to be transmitted recorded in the transmission event queue;

[0018] According to the file descriptor, obtaining the triggering status of the event data to be transmitted;

[0019] When the triggering state of the event data to be transmitted is a successful trigger, the event data to be transmitted meets a preset transmission condition.

[0020] In a possible implementation, the method further includes:

[0021] When the data to be transmitted is updated, obtaining data update information of the data to be transmitted;

[0022] Based on the data update information, the data transmission queue is updated.

[0023] In a possible implementation manner, the data update information includes event data update information and file data update information;

[0024] The updating of the data transmission queue based on the data update information includes:

[0025] Based on the event data update information, the to-be-transmitted event queue is updated, so as to update the data transmission queue based on the updated to-be-transmitted event queue;

[0026] Based on the file data update information, the data transmission queue is updated.

[0027] In a possible implementation, updating the queue of events to be transmitted based on the event data update information includes:

[0028] Determining an update status of the event data to be transmitted according to the event data update information;

[0029] When the update status is new, an event new queue is established according to the event data update information, and the new event data recorded in the event new queue is sequentially added to the event queue to be transmitted, so as to implement the update of the event queue to be transmitted;

[0030] In the case where the update status is deletion, establishing an event deletion queue according to the event data update information, and determining the event data to be deleted in the event queue to be transmitted that corresponds to the event deletion queue;

[0031] The event data to be deleted recorded in the event queue to be transmitted are deleted in sequence to update the event queue to be transmitted.

[0032] In a possible implementation, updating the data transmission queue based on the file data update information includes:

[0033] Determining an update status of the file data to be transmitted according to the file data update information;

[0034] In the case where the update status is newly added, determining a newly added file according to the file data update information, and adding the newly added file to the data transmission queue;

[0035] When the update status is deletion, the to-be-deleted files are determined according to the file data update information, and the to-be-deleted files in the data transmission queue are deleted.

[0036] In a second aspect, the present application provides a data transmission device, the device comprising:

[0037] A first acquisition module, used to acquire data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted;

[0038] A sending module, used for sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions among the data to be transmitted to the data transmission queue;

[0039] A second acquisition module is used to acquire a mapping relationship between a file descriptor of each data to be transmitted and a multiplexing descriptor recorded in the data transmission queue;

[0040] The transmission module is used to sequentially transmit the target transmission data in the data transmission queue according to the mapping relationship.

[0041] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the data transmission method described in any one of the embodiments of the first aspect.

[0042] In a fourth aspect, the present application further provides a computer storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the data transmission method described in any embodiment of the first aspect are implemented.

[0043] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the data transmission method provided by the embodiment of the present application obtains the data to be transmitted through a public information channel without occupying thread resources separately, thus saving CPU thread resources; and realizes efficient data processing and transmission through the mapping relationship between file descriptors and multiplexing descriptors, while reducing the consumption of system resources; in addition, it can also improve the stability and reliability of data transmission, and the use of multiplexing technology can process multiple input and output operations at the same time, avoiding the waste of resources caused by thread blocking. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0047] Figure 1 A flowchart of a data transmission method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the steps of a data transmission method provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the structure of a data transmission device provided in an embodiment of the present application;

[0050] Figure 4A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0052] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0053] In order to solve the problem in the prior art that each connection in multiple connections needs to ensure low-latency processing and exclusive CPU thread resources, while CPU resources are limited and more CPU resources and other system resources are needed, which undoubtedly poses a huge challenge to the performance of the server and leads to the technical problem of overall low efficiency, the present application provides a data transmission method, device, electronic device and storage medium, which obtains the data to be transmitted through a public information channel without occupying thread resources separately, thereby effectively saving CPU thread resources. At the same time, through the mapping relationship between the file descriptor and the multiplexing descriptor, efficient data processing and transmission are achieved, which not only improves the efficiency of data processing and transmission, but also significantly reduces the consumption of system resources. In addition, the use of multiplexing technology can process multiple input and output operations at the same time, which greatly improves the stability and reliability of data transmission, avoids the waste of resources caused by thread blocking, and ensures the efficient operation of the system.

[0054] Figure 1 A flow chart of a data transmission method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method specifically includes:

[0055] S101, acquiring data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted;

[0056] In this embodiment, the data to be transmitted is obtained through the public information channel, and the data to be transmitted includes two categories: one is the event data to be transmitted, and the other is the file data to be transmitted. Specifically, the event data to be transmitted refers to the event information that needs to be transmitted in the system or network, such as various status updates, operation records, log information, etc. The file data to be transmitted covers various types of files, such as documents, pictures, audio, video, etc. The files to be transmitted need to be transmitted in the network for use or storage by the recipient.

[0057] The public information channel can process events of multiple connections simultaneously, providing a low-latency public event channel shared by multiple connections, which can be achieved with extremely small computing resources.

[0058] S102, sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions among the data to be transmitted to a data transmission queue;

[0059] In this embodiment, the data to be transmitted is analyzed to determine whether the data to be transmitted meets the preset transmission conditions, and the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions are sent to the data transmission queue respectively to ensure the efficiency and accuracy of data transmission.

[0060] Furthermore, the data transmission queue is responsible for managing the data to be transmitted and scheduling it according to priority and time order to ensure that the data to be transmitted can be transmitted according to established rules; in addition, the data transmission queue also has a caching function, which can temporarily store data that cannot be transmitted immediately to prevent data loss; during the data transmission process, the transmission strategy can be dynamically adjusted according to the network conditions to adapt to different network environments and ensure the stability and reliability of data transmission.

[0061] S103, obtaining a mapping relationship between a file descriptor of each data to be transmitted and a multiplexing descriptor recorded in the data transmission queue;

[0062] The file descriptor is used to uniquely identify the file data in the data to be transmitted; through the file descriptor, the data transmission device can quickly locate the specific data file, thereby improving the efficiency of data processing; at the same time, the multiplexing descriptor is used to describe the transmission status of multiple data streams, so that the device can manage multiple data transmission tasks at the same time and achieve efficient data transmission.

[0063] In this embodiment, the mapping relationship between the file descriptor and the multiplexing descriptor corresponding to each data to be transmitted in the data transmission queue is obtained, and the mapping relationship can be quickly retrieved and updated through an efficient data structure to ensure that various operation requests can be responded to quickly during the data transmission process.

[0064] S104. According to the mapping relationship, sequentially transmit the target transmission data in the data transmission queue.

[0065] In this embodiment, after the mapping relationship is determined, the target transmission data is taken out from the data transmission queue, and is transmitted in sequence according to the previously determined mapping relationship, thereby ensuring the correctness of the data and the efficiency of the transmission.

[0066] Specifically, the target transmission data in the data transmission queue is transmitted sequentially according to the order recorded in the data transmission queue, and the data to be transmitted is sent to the target location in turn; the sequentiality of data transmission is guaranteed, and due to the existence of the mapping relationship, a fast conversion from file descriptor to multiplexed descriptor can be achieved, thereby speeding up the data transmission speed.

[0067] In addition, due to the priority and time sequence management of the data transmission queue, it can ensure that high-priority data can be transmitted first, further optimizing the efficiency and response speed of data transmission.

[0068] During the data transmission process, this embodiment also takes into account the dynamic changes in network conditions and can adjust the transmission strategy according to the real-time network conditions. For example, when the network conditions are poor, the data transmission rate can be reduced to reduce the loss of data packets; when the network conditions are good, the transmission rate can be increased to make full use of network resources. This adaptive transmission strategy not only improves the stability of data transmission, but also improves the overall transmission efficiency.

[0069] The data transmission method provided in the embodiment of the present application first obtains the data to be transmitted through a public information channel. The data to be transmitted is obtained through the public information channel without occupying thread resources separately, thus saving CPU thread resources; then the event data to be transmitted and the file data to be transmitted in the data to be transmitted are sent to the data transmission queue respectively, and then the mapping relationship between the file descriptor and the multiplexing descriptor of each data to be transmitted in the data transmission queue is obtained to sequentially transmit the data, thereby reducing the consumption of system resources and improving the stability and reliability of data transmission. The multiplexing technology can process multiple input and output operations at the same time, avoiding the waste of resources caused by thread blocking, making the data transmission process more efficient and stable, and also greatly improving the overall performance of the system.

[0070] In an optional solution of the embodiment of the present invention, sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission condition in the data to be transmitted to the data transmission queue includes:

[0071] Sending the event data to be transmitted included in the data to be transmitted to a queue of events to be transmitted;

[0072] Determine whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition, and obtain a determination result;

[0073] According to the judgment result, the queue of events to be transmitted is updated so that the event data recorded in the queue of events to be transmitted is the event data to be transmitted that meets the preset transmission condition;

[0074] The event data to be transmitted and the file data to be transmitted recorded in the event queue to be transmitted are sent to the data transmission queue.

[0075] The data to be transmitted includes two different data types, namely, event data to be transmitted and file data to be transmitted. File data to be transmitted can be directly added to the data transmission queue for subsequent data transmission operations. However, for event data to be transmitted, it is necessary to determine whether the corresponding event has been triggered. Only when the corresponding event is triggered can the event data to be transmitted be added to the data transmission queue and the data transmission work be started.

[0076] In this embodiment, first, the event data to be transmitted contained in the data to be transmitted is sent to the event queue to be transmitted. The event queue to be transmitted will be used to store and manage all event data to be transmitted. Then the event queue to be transmitted is checked to determine whether the event data to be transmitted recorded therein meets the preset transmission conditions. The checking process involves detailed analysis and evaluation of each event data in the queue to ensure that the event data meets the transmission requirements.

[0077] After completing the judgment of the event data, the judgment result is obtained. According to the judgment result, the event queue to be transmitted is updated, and the event data that does not meet the conditions in the event queue to be transmitted is removed or marked as pending, so as to ensure that the event data recorded in the queue meets the preset transmission conditions, and only the event data that meets the conditions will be retained in the queue.

[0078] Finally, the event data to be transmitted that meets the preset transmission conditions recorded in the event queue to be transmitted is sent to the data transmission queue together with the file data to be transmitted. The data transmission queue will be responsible for managing and executing the actual data transmission process to ensure that all qualified event data and file data can be smoothly transmitted from the source end to the target end.

[0079] In an optional solution of the embodiment of the present invention, the determining whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition to obtain a determination result includes:

[0080] Obtaining the file descriptor of the event data to be transmitted recorded in the transmission event queue;

[0081] According to the file descriptor, obtaining the triggering status of the event data to be transmitted;

[0082] When the triggering state of the event data to be transmitted is a successful trigger, the event data to be transmitted meets a preset transmission condition.

[0083] In this embodiment, firstly, the file descriptor of the recorded event data to be transmitted is extracted from the transmission event queue. The file descriptor is a unique identifier used to locate and operate a specific file or data stream in the system.

[0084] Then, the current triggering state of the event data to be transmitted is further obtained through the file descriptor, so as to determine whether the event data is ready for transmission and whether the predetermined transmission conditions are met.

[0085] Finally, when it is confirmed that the trigger status of the event data to be transmitted is successfully triggered, that is, the event data has met the preset transmission conditions (data integrity check, security verification or other relevant requirements), the smooth transmission of data is ensured, thereby ensuring the accuracy and reliability of the data.

[0086] For example, by using the two interfaces select or epoll_wait, an efficient multiplexed event processing mechanism is implemented. During the implementation process, an event queue named ep is maintained internally, which is responsible for storing all triggered events. The triggering and adding of events are completed through an internal real-time polling mechanism. The polling mechanism will continuously check the public event channel to determine whether there are new events. Once the event is triggered, the event data is added to the ep event queue. The user of the interface only needs to extract events from this ep event queue and complete the corresponding interface parameter docking work to easily realize the triggering of multiplexed events. The algorithm complexity of this processing method is O(n), which means that its performance is linearly related to the number of events, thereby ensuring extremely high real-time performance.

[0087] At the upper level of the system, the daemon process will continuously loop through the public event channel. By consuming the computing resources of a CPU core, the daemon process can achieve low-latency processing of public events to detect whether any events are triggered, ensuring that various events can be responded to quickly, thereby improving the overall system performance and user experience.

[0088] In an optional solution of the embodiment of the present invention,

[0089] When the data to be transmitted is updated, obtaining data update information of the data to be transmitted;

[0090] Based on the data update information, the data transmission queue is updated.

[0091] In this embodiment, when the data to be transmitted changes, the system will automatically detect the data change and trigger an update event, which will carry detailed data update information, including the type of data update (such as addition, modification or deletion), updated data content and updated timestamp, etc. The update information can accurately identify which data has changed and track the specific change time.

[0092] Specifically, check the data in the data transmission queue to ensure that the data in the queue is the latest data; if it is found that the data in the queue is outdated, the system will automatically replace it with the latest data, so that the data transmission queue is always kept up to date, thereby improving the real-time and accuracy of data processing.

[0093] In addition, by ensuring that the data in the data transmission queue is up to date, the present application also ensures the real-time and consistency of data transmission; no matter when and where the data update occurs, it can respond quickly and perform corresponding update operations to ensure the real-time nature of data transmission.

[0094] Exemplarily, the addition and deletion operations of the multiplexed socket are implemented by using the interface file FD_SET or epoll_ctl. Specifically, when a new socket file descriptor is added to the multiplexed socket, this function can be implemented by calling the corresponding interface function. On the contrary, when a socket file descriptor is removed from the multiplexed socket, this operation can also be completed by calling the corresponding interface function. In the internal implementation, all currently active socket file descriptors are stored by maintaining the file descriptor array. When a new socket file descriptor needs to be added, the corresponding element is added to the array; and when a socket file descriptor needs to be deleted, the corresponding element is removed from the array; thereby flexibly managing the socket file descriptors in the multiplexed socket, thereby achieving efficient network communication.

[0095] In an optional solution of the embodiment of the present invention, the data update information includes event data update information and file data update information;

[0096] The updating of the data transmission queue based on the data update information includes:

[0097] Based on the event data update information, the to-be-transmitted event queue is updated, so as to update the data transmission queue based on the updated to-be-transmitted event queue;

[0098] Based on the file data update information, the data transmission queue is updated.

[0099] In this embodiment, during the data update process, two different situations are distinguished: event data update and file data update. For the update of event data, it is necessary to update the queue of events to be transmitted to ensure the smooth progress of data transmission. First, it is necessary to modify the list of events to be transmitted to ensure that all the latest event data are taken into consideration. Then, based on the updated list of events to be transmitted, the data transmission queue is further updated. The update of file data is relatively direct. The file data can be directly updated by updating the data transmission queue. That is, when there are new or modified files to be transmitted, these file data are directly included in the data transmission queue. There is no need to manage the file data through an intermediate list, but they can be directly transmitted. By processing the updates of event data and file data separately, different types of data can be effectively managed, and efficient and accurate data transmission can be ensured.

[0100] In an optional solution of the embodiment of the present invention, updating the queue of events to be transmitted based on the event data update information includes:

[0101] Determining an update status of the event data to be transmitted according to the event data update information;

[0102] When the update status is new, an event new queue is established according to the event data update information, and the new event data recorded in the event new queue is sequentially added to the event queue to be transmitted, so as to implement the update of the event queue to be transmitted;

[0103] In the case where the update status is deletion, establishing an event deletion queue according to the event data update information, and determining the event data to be deleted in the event queue to be transmitted that corresponds to the event deletion queue;

[0104] The event data to be deleted recorded in the event queue to be transmitted are deleted in sequence to update the event queue to be transmitted.

[0105] In this embodiment, the update status of the event data to be transmitted is clarified according to the provided event data update information. Specifically, it is determined whether the updated event data is newly added data or data to be deleted;

[0106] When the update status is determined to be new, an event new queue is constructed based on the event data update information. Each new event data is recorded in the event new queue, and these new event data are added to the queue of events to be transmitted in a certain order. In this way, the queue of events to be transmitted is ensured to be updated in a timely and accurate manner.

[0107] When the update status is determined to be deletion, an event deletion queue is created based on the event data update information. The event deletion queue records all event data that need to be deleted. When deleting data, the event data to be deleted corresponding to the event deletion queue is found in the event queue to be transmitted, and the deletion operation is performed in sequence, thereby realizing the update of the event queue to be transmitted. Through this deletion process, it is ensured that the event queue to be transmitted no longer contains data that is no longer needed, thereby maintaining the accuracy and timeliness of the queue.

[0108] In an optional solution of the embodiment of the present invention, updating the data transmission queue based on the file data update information includes:

[0109] Determining an update status of the file data to be transmitted according to the file data update information;

[0110] In the case where the update status is newly added, determining a newly added file according to the file data update information, and adding the newly added file to the data transmission queue;

[0111] When the update status is deletion, the to-be-deleted files are determined according to the file data update information, and the to-be-deleted files in the data transmission queue are deleted.

[0112] In this embodiment, the update status of the file data to be transmitted is determined according to the provided file data update information. Specifically, the latest status of the file data is analyzed to determine whether a corresponding update operation is required.

[0113] When the update status is new, it is necessary to determine the new files based on the file data update information, and then add the new files to the data transmission queue. All files to be transmitted are transmitted through the data transmission queue to ensure that the data is transmitted in a predetermined order and priority.

[0114] When the update status is deletion, the files to be deleted are determined according to the file data update information, and then removed from the data transmission queue, ensuring that the data transmission queue contains only valid files that need to be transmitted, avoiding unnecessary data transmission and potential errors.

[0115] Figure 2 A schematic diagram of the steps of a data transmission method provided in an embodiment of the present application is as follows: Figure 2 As shown, the data transmission steps are as follows:

[0116] Step 1: First, create a socket, which is the same as the standard socket implementation. By calling the socket() function interface, a corresponding sock file descriptor is generated for subsequent network communication operations.

[0117] Step 2: Create a multiplexed file descriptor, which is equivalent to initializing the multiplexed file descriptor. Specifically, a queue operation needs to be created so that multiple file descriptors can be managed and operated uniformly.

[0118] Step 3: By calling the select or epoll_ctrl interface, the sock file descriptor is associated with the multiplexed file descriptor, that is, the sock file descriptor is added to the corresponding multiplexed queue, so that it can be queried from the queue later to see if there is an event trigger, so as to perform corresponding processing.

[0119] Step 4: The hardware network card senses external event changes, such as new connection requests or new data to be received on a connected connection. When these events occur, the hardware network card will process them accordingly and transfer the processing results to the DMA public event channel. The public event channel contains all necessary event notification messages for all connections for subsequent processing.

[0120] Step 5: The upper-level daemon process continuously loops through the public event channel to detect whether there is an event trigger. This process consumes the computing power of a CPU core to achieve the effect of low-latency processing of public events, thereby improving the response speed and performance of the system.

[0121] Step 6: Check whether the connection that triggers the event is associated with the multiplexed queue. If it is, the event needs to be added to the multiplexed event queue so that these events can be processed in a timely manner and the processing efficiency of the system can be improved.

[0122] Step 7: By calling the select or epoll_wait interface, the events in the event queue are obtained in time and fed back to the caller in time, so that the caller can quickly know which events have been triggered and perform corresponding processing.

[0123] Step 8: In the data sending and receiving processing stage, each connection has its own sending and receiving data queue. At this time, the data and events are synchronized, that is, when the event feedback arrives, the corresponding data is already in an operational state, ensuring the accuracy and efficiency of data transmission.

[0124] In the data transmission method of the above example, in the data sending and receiving processing stage, each connection has its own sending and receiving data queue, and the data and events are synchronized, that is, when the event feedback arrives, the corresponding data is already in an operational state, ensuring the accuracy and efficiency of data transmission, realizing an efficient and stable data transmission process, and greatly improving the efficiency of data processing and the overall performance of the system.

[0125] Figure 3A schematic diagram of a data transmission device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the device specifically includes:

[0126] A first acquisition module 301 is used to acquire data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted;

[0127] The sending module 302 is used to send the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions in the data to be transmitted to the data transmission queue;

[0128] A second acquisition module 303 is used to acquire a mapping relationship between a file descriptor of each data to be transmitted and a multiplexing descriptor recorded in the data transmission queue;

[0129] The transmission module 304 is used to sequentially transmit the target transmission data in the data transmission queue according to the mapping relationship.

[0130] In one possible implementation, the sending module 302 is also used to send the event data to be transmitted included in the data to be transmitted to the event queue to be transmitted; determine whether the event data to be transmitted recorded in the transmission event queue meets the preset transmission conditions to obtain a determination result; update the event queue to be transmitted based on the determination result so that the event data recorded in the event queue to be transmitted is the event data to be transmitted that meets the preset transmission conditions; and send the event data to be transmitted and the file data to be transmitted recorded in the event queue to be transmitted to the data transmission queue.

[0131] In a possible implementation, the sending module 302 is also used to obtain the file descriptor of the event data to be transmitted recorded in the transmission event queue; based on the file descriptor, obtain the trigger status of the event data to be transmitted; when the trigger status of the event data to be transmitted is a successful trigger, the event data to be transmitted meets the preset transmission conditions.

[0132] In a possible implementation manner, the device further includes an updating module 305, which is used to obtain data update information of the data to be transmitted if there is an update to the data to be transmitted; and update the data transmission queue based on the data update information.

[0133] In a possible implementation, the update module 305 is also used to update the data transmission queue based on the data update information, including: updating the event queue to be transmitted based on the event data update information, so as to update the data transmission queue based on the updated event queue to be transmitted; updating the data transmission queue based on the file data update information.

[0134] In one possible implementation, the update module 305 is also used to determine the update status of the event data to be transmitted based on the event data update information; when the update status is new addition, an event addition queue is established based on the event data update information, and the new event data recorded in the event addition queue is added to the event queue to be transmitted in sequence, so as to realize the update of the event queue to be transmitted; when the update status is deletion, an event deletion queue is established based on the event data update information, and the event data to be deleted in the event queue to be transmitted corresponding to the event deletion queue is determined; and the event data to be deleted recorded in the event queue to be transmitted is deleted in sequence, so as to realize the update of the event queue to be transmitted.

[0135] In one possible implementation, the update module 305 is also used to determine the update status of the file data to be transmitted based on the file data update information; when the update status is new addition, the new file is determined based on the file data update information, and the new file is added to the data transmission queue; when the update status is deletion, the file to be deleted is determined based on the file data update information, and the file to be deleted in the data transmission queue is deleted.

[0136] The system security scanning device provided in this embodiment can be as follows Figure 3 The data transmission device shown in , can execute as follows Figure 1-2 All steps of data transmission in Figure 1-2 For details on the technical effects of data transmission shown, please refer to Figure 1-2 For the sake of brevity, the relevant description is not repeated here.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404; the memory 403 is used to store computer programs; the processor 401 is used to execute the program stored in the memory 403, and implement the steps of the data transmission method provided by any of the aforementioned method embodiments:

[0139] Acquire data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted; send the event data to be transmitted and the file data to be transmitted that meet preset transmission conditions in the data to be transmitted to a data transmission queue; acquire a mapping relationship between a file descriptor and a multiplexing descriptor of each data to be transmitted recorded in the data transmission queue; and sequentially transmit the target transmission data in the data transmission queue according to the mapping relationship.

[0140] In one possible implementation, the event data to be transmitted included in the data to be transmitted is sent to an event queue to be transmitted; it is determined whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition to obtain a determination result; based on the determination result, the event queue to be transmitted is updated so that the event data recorded in the event queue to be transmitted is the event data to be transmitted that meets the preset transmission condition; the event data to be transmitted and the file data to be transmitted recorded in the event queue to be transmitted are sent to a data transmission queue.

[0141] In one possible implementation, a file descriptor of the event data to be transmitted recorded in the transmission event queue is obtained; based on the file descriptor, a trigger status of the event data to be transmitted is obtained; when the trigger status of the event data to be transmitted is a successful trigger, the event data to be transmitted meets a preset transmission condition.

[0142] In a possible implementation, when the data to be transmitted is updated, data update information of the data to be transmitted is acquired; and based on the data update information, the data transmission queue is updated.

[0143] In a possible implementation, updating the data transmission queue based on the data update information includes: updating the event queue to be transmitted based on the event data update information, so as to update the data transmission queue based on the updated event queue to be transmitted; and updating the data transmission queue based on the file data update information.

[0144] In one possible implementation, the update status of the event data to be transmitted is determined based on the event data update information; when the update status is new addition, an event addition queue is established based on the event data update information, and the new event data recorded in the event addition queue is added to the event queue to be transmitted in sequence, so as to achieve the update of the event queue to be transmitted; when the update status is deletion, an event deletion queue is established based on the event data update information, and the event data to be deleted in the event queue to be transmitted corresponding to the event deletion queue is determined; and the event data to be deleted recorded in the event queue to be transmitted is deleted in sequence, so as to achieve the update of the event queue to be transmitted.

[0145] In one possible implementation, the update status of the file data to be transmitted is determined based on the file data update information; when the update status is new addition, the new file is determined based on the file data update information, and the new file is added to the data transmission queue; when the update status is deletion, the file to be deleted is determined based on the file data update information, and the file to be deleted in the data transmission queue is deleted.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0147] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0148] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data transmission method, characterized in that: include: Acquire data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted; Sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions among the data to be transmitted to the data transmission queue; Obtaining a mapping relationship between a file descriptor of each data to be transmitted and a multiplexing descriptor recorded in the data transmission queue; According to the mapping relationship, the target transmission data in the data transmission queue is transmitted sequentially.

2. The method according to claim 1, characterized in that The step of sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions in the data to be transmitted to the data transmission queue includes: Sending the event data to be transmitted included in the data to be transmitted to a queue of events to be transmitted; Determine whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition, and obtain a determination result; According to the judgment result, the queue of events to be transmitted is updated so that the event data recorded in the queue of events to be transmitted is the event data to be transmitted that meets the preset transmission condition; The event data to be transmitted and the file data to be transmitted recorded in the event queue to be transmitted are sent to the data transmission queue.

3. The method according to claim 2, characterized in that The determining whether the event data to be transmitted recorded in the transmission event queue meets a preset transmission condition to obtain a determination result includes: Obtaining the file descriptor of the event data to be transmitted recorded in the transmission event queue; According to the file descriptor, obtaining the triggering status of the event data to be transmitted; When the triggering state of the event data to be transmitted is a successful trigger, the event data to be transmitted meets a preset transmission condition.

4. The method according to claim 1, characterized in that The method further comprises: When the data to be transmitted is updated, obtaining data update information of the data to be transmitted; Based on the data update information, the data transmission queue is updated.

5. The method according to claim 4, characterized in that The data update information includes event data update information and file data update information; The updating of the data transmission queue based on the data update information includes: Based on the event data update information, the to-be-transmitted event queue is updated, so as to update the data transmission queue based on the updated to-be-transmitted event queue; Based on the file data update information, the data transmission queue is updated.

6. The method according to claim 5, characterized in that The updating of the to-be-transmitted event queue based on the event data update information includes: Determining an update status of the event data to be transmitted according to the event data update information; When the update status is new, an event new queue is established according to the event data update information, and the new event data recorded in the event new queue is sequentially added to the event queue to be transmitted, so as to implement the update of the event queue to be transmitted; In the case where the update status is deletion, establishing an event deletion queue according to the event data update information, and determining the event data to be deleted in the event queue to be transmitted that corresponds to the event deletion queue; The event data to be deleted recorded in the event queue to be transmitted are deleted in sequence to update the event queue to be transmitted.

7. The method according to claim 5, characterized in that The updating of the data transmission queue based on the file data update information includes: Determining an update status of the file data to be transmitted according to the file data update information; In the case where the update status is newly added, determining a newly added file according to the file data update information, and adding the newly added file to the data transmission queue; When the update status is deletion, the to-be-deleted files are determined according to the file data update information, and the to-be-deleted files in the data transmission queue are deleted.

8. A data transmission device, characterized in that: include: A first acquisition module, used to acquire data to be transmitted through a public information channel, wherein the data to be transmitted includes event data to be transmitted and file data to be transmitted; A sending module, used for sending the event data to be transmitted and the file data to be transmitted that meet the preset transmission conditions among the data to be transmitted to the data transmission queue; A second acquisition module is used to acquire a mapping relationship between a file descriptor of each data to be transmitted and a multiplexing descriptor recorded in the data transmission queue; The transmission module is used to sequentially transmit the target transmission data in the data transmission queue according to the mapping relationship.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the data transmission method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 7 are implemented.

Images