Routing oscillation positioning method and apparatus, non-volatile readable storage medium, and electronic device
By establishing the correspondence between the target read and write tasks and the target read and write events, the task request is directly processed when the target read and write events are triggered, solving the problems of large resource consumption and low processing efficiency in a large number of concurrent scenarios, and achieving efficient asynchronous read and write processing.
Patent Information
- Application Number
- PCT/CN2024/084589
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-28
- Filing Date
- 2024-03-28
- Publication Date
- 2025-03-06
AI Technical Summary
In the case of large-scale concurrency, the task initiator consumes a large amount of resources and low processing efficiency. The existing asynchronous I/O methods require processing of I/O return, occupying resources, and are not suitable for a large-scale concurrent scenario.
By establishing the correspondence between the target read and write tasks and the target read and write events, the target read and write tasks are placed into the target read and write events. When the target read and write events are triggered, the target read and write task requests are directly processed to avoid returning data to the task initiator for processing.
It improves processing efficiency, reduces resource consumption, is suitable for a large number of concurrent scenarios, and realizes the response and processing of asynchronous read and write task requests.
Smart Images

Figure CN2024084589_06032025_PF_FP_ABST
Abstract
Description
[Corrected 19.04.2024 according to Rule 91] Asynchronous reading and writing method, device, computer equipment and non-volatile readable storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] [Corrected on 19.04.2024 according to Rule 91] This application claims priority to the Chinese patent application filed with the Chinese Patent Office on August 28, 2023, with application number 202311085817.9 and application name “Asynchronous reading and writing method, device, computer equipment and storage medium”, all contents of which are incorporated by reference into this application. Technical Field
[0003] The embodiments of the present application relate to the field of data reading and writing technology, and in particular, to asynchronous reading and writing methods, devices, computer equipment, and non-volatile readable storage media. Background Art
[0004] Asynchronous I / O (Input / Output) is an I / O method that allows other calculations before data transmission is completed. It returns immediately after initiating an I / O request and does not notify the application in some form until the kernel completes the I / O request. To implement asynchronous I / O, you can usually set the file read and write mode to non-blocking mode or use the select() interface. However, using the above two methods, the I / O requester also needs to process the I / O return. Whether using notification or synchronous waiting to obtain the I / O return, it will take up a certain amount of resource consumption, increase the request processing process, and is not suitable for scenarios with a large amount of concurrency.
[0005] Summary of the Invention
[0006] In view of this, the embodiments of the present application provide an asynchronous reading and writing method, apparatus, computer equipment and non-volatile readable storage medium to solve the problem of large resource consumption and low processing efficiency at the task initiator in a large number of concurrent situations.
[0007] In a first aspect, an embodiment of the present application provides an asynchronous reading and writing method, the method comprising:
[0008] Based on the tasks included in the read and write task request, create the target read and write tasks corresponding to the tasks respectively;
[0009] Put the target read and write task into the target read and write event; the target read and write task corresponds to the target read and write event one by one;
[0010] Determine the state of the target read / write event based on the trigger data set; the trigger data set includes trigger data of the read / write event in the triggered state;
[0011] When the state of the target read / write event is a trigger state, based on the target read / write event, the target read / write task corresponding to the target read / write event is processed.
[0012] In this way, when the target read / write event is triggered, the read / write task request can be processed directly according to the target read / write event, without returning the data in the target read / write event to the task initiator for further processing, thereby improving processing efficiency and reducing resource consumption.
[0013] In an optional implementation, based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are established, including:
[0014] Determine the task configuration information for each task in the read / write task request;
[0015] Based on the task configuration information of each task, the target read and write tasks corresponding to each task are established respectively.
[0016] In an optional implementation, based on the task configuration information of each task, a target read and write task corresponding to each task is established, including:
[0017] Determine the task identification information of the task as the target identification information of the target read-write task corresponding to the task;
[0018] The task callback function of the task is determined as the target callback function of the target read-write task corresponding to the task.
[0019] In this way, the relevant information of the read / write task request can be synchronized to the target read / write task, and the response and processing of the read / write task request can be realized through the target read / write task.
[0020] In an optional implementation, placing the target read / write task into the target read / write event includes:
[0021] Determine the target read and write events corresponding to the target read and write tasks;
[0022] Write the target read / write task into the event data of the target read / write event.
[0023] In this way, the target read and write task can be used as part of the event data of the target read and write event. When the target read and write event is triggered and the event data of the target read and write event is obtained, the event data is processed by the target read and write task contained in the event data, thereby completing the response and processing of the read and write task request.
[0024] In an optional implementation, determining a target read / write event corresponding to a target read / write task includes:
[0025] Using the target identification information of the target read / write task as the event identification information of the target read / write event;
[0026] Determines the event type information of the target read and write events.
[0027] In this way, the target read / write task and the target read / write event can be matched through the target identification information of the target read / write task and the event identification information of the target read / write event.
[0028] In an optional embodiment, the method further includes:
[0029] Based on the task control function, the task instance and the target read / write task are associated to add the target read / write event to the event collection of the task instance.
[0030] In an optional implementation, based on the task control function, the task instance and the target read / write task are associated to add the target read / write event to the event set of the task instance, including:
[0031] Set the instance identifier in the task control function to the instance identifier information of the task instance;
[0032] Set the task ID in the task control function to the target ID information of the target read / write task;
[0033] Set the operation parameter in the task control function to the value corresponding to the added event;
[0034] Call the task control function to associate the task instance with the target read / write task and add the target read / write event to the event collection of the task instance.
[0035] In this way, the task instance and the target read / write task can be associated through the task control function, thereby adding the target read / write event to the event collection of the task instance.
[0036] In an optional implementation, determining the state of the target read / write event based on the trigger data set includes:
[0037] Get the trigger data set;
[0038] When the event identification information in the trigger data set is consistent with the target identification information of the target read / write task, the state of the target read / write event is determined to be the trigger state.
[0039] In this way, it can be determined from the trigger data set whether the read / write events in the trigger state include the target read / write event according to the event identification information.
[0040] In an optional embodiment, the method further includes:
[0041] When the data to be processed is written into the event data of the target read / write event, the event type information and event identification information of the target read / write event are used as trigger data of the target read / write event;
[0042] Add the trigger data of the target read / write event to the trigger data set.
[0043] In this way, when the data to be processed is written into the target read / write event, the target read / write event is triggered.
[0044] In an optional embodiment, after adding the trigger data of the target read / write event to the trigger data set, the method further includes:
[0045] Clear the event type information in the target read and write events.
[0046] In an optional implementation, based on the target read / write event, processing the read / write task request corresponding to the target read / write event includes:
[0047] Get the event data of the target read and write events;
[0048] Extract target read / write tasks and data to be processed from event data;
[0049] Based on the target reading and writing tasks, the data to be processed is processed.
[0050] In this way, when a target read / write event is triggered, the data to be processed can be processed by the target read / write task contained in the event data, without the need to retrieve the data to be processed and return it to the task initiator, thereby improving processing efficiency.
[0051] In an optional implementation, processing the data to be processed based on the target read / write task includes:
[0052] Get the target callback function in the target read and write task;
[0053] Call the target callback function to process the data to be processed.
[0054] In this way, the data to be processed can be processed according to the target callback function corresponding to the read / write task request, thereby completing the data processing according to the requirements of the read / write task request.
[0055] In an optional implementation, after processing the read / write task request based on the target read / write event, the method further includes:
[0056] When the number of responses to the read / write task request reaches the preset number of responses, the association between the task instance and the target read / write task is released.
[0057] In an optional implementation, disassociating the task instance from the target read / write task includes:
[0058] Set the operation parameter in the task control function to the value corresponding to the deletion event;
[0059] Call the task control function to disassociate the task instance from the target read / write task and delete the target read / write event from the event collection of the task instance.
[0060] In an optional implementation, after processing the read / write task request based on the target read / write event, the method further includes:
[0061] When the number of responses to the read / write task request does not reach the preset number of responses, the target read / write event is modified to supplement the event type information in the target read / write event.
[0062] In an optional implementation, the target read / write event is modified to supplement the event type information in the target read / write event, including:
[0063] Determine the event type information of the target read and write event;
[0064] Set the operation parameter in the task control function to the value corresponding to the modification event;
[0065] Call the task control function to modify the target read and write event to supplement the event type information in the target read and write event.
[0066] In this way, when the number of responses to the read / write task request reaches the required preset number of responses, the association between the task instance and the target read / write task can be released, avoiding repeated establishment and release of the association between the task instance and the target read / write task, thereby improving processing efficiency.
[0067] In an optional embodiment, the method further includes:
[0068] When the task waiting function returns the first preset value, it is determined that the read / write task request is not responded to.
[0069] In an optional embodiment, the method further includes:
[0070] When the target read / write task corresponding to each task in the read / write task request is processed completely, it is determined that the read / write task request response is complete.
[0071] In this way, the read-write task request can be split into multiple tasks, and the tasks contained in the read-write task request can be processed separately. When each task in the read-write task request is processed, it is determined that the read-write task request response is completed. The read-write task request no longer needs to occupy the process and wait for the return and processing of data, thereby realizing the asynchronous response of the read-write task request.
[0072] In a second aspect, an embodiment of the present application provides an asynchronous read / write device, the device comprising:
[0073] The target reading and writing task establishment module is configured to establish target reading and writing tasks corresponding to the tasks respectively based on the tasks included in the reading and writing task request;
[0074] The target read / write event writing module is configured to put the target read / write task into the target read / write event; the target read / write task corresponds to the target read / write event in a one-to-one manner;
[0075] The read / write event state determination module is configured to determine the state of a target read / write event based on a trigger data set; the trigger data set includes trigger data of a read / write event in a triggered state;
[0076] The read / write task request processing module is configured to process a target read / write task corresponding to the target read / write event based on the target read / write event when the state of the target read / write event is a trigger state.
[0077] In a third aspect, an embodiment of the present application provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the asynchronous reading and writing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0078] In a fourth aspect, an embodiment of the present application provides a non-volatile readable storage medium having computer instructions stored thereon, and the computer instructions are configured to enable a computer to execute the asynchronous reading and writing method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] In order to more clearly illustrate the optional implementation methods of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the optional implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0080] FIG1 is a flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0081] FIG2 is a second flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0082] FIG3 is a third flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0083] FIG4 is a fourth flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0084] FIG5 is a fifth flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0085] FIG6 is a sixth flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0086] FIG7 is a seventh flow chart of an asynchronous reading and writing method according to an embodiment of the present application;
[0087] FIG8 is a schematic diagram of thread flow based on the asynchronous read and write method provided in this example;
[0088] FIG9 is a block diagram of an asynchronous read / write device according to an embodiment of the present application;
[0089] FIG10 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION
[0090] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the embodiments of the present application.
[0091] In Linux, I / O (Input / Output) can be categorized into five modes: blocking I / O, non-blocking I / O, I / O multiplexing, signal-driven I / O, and asynchronous I / O. Blocking I / O interfaces block the current thread until the system call returns a result or a timeout error occurs. High concurrency requires the creation of a large number of threads to maintain network connections, resulting in significant memory and thread overhead. Non-blocking I / O interfaces require the user process to continuously confirm data availability with the system after initiating a request, thus consuming significant resources. Multiplexed I / O interfaces require constant polling of all their assigned sockets. This can consume significant resources and time when there are many sockets involved. Signal-driven I / O interfaces utilize signaling mechanisms to prevent the calling thread from blocking, but they still require waiting for data to be copied, resulting in low processing efficiency. The above four I / O interface methods all have the defects of high resource consumption or low processing efficiency, and are not suitable for situations with large concurrency. The asynchronous I / O interface, on the other hand, can support I / O that allows other calculations before data transmission is completed. It returns immediately after initiating an I / O request and notifies the application in some form after the kernel completes the I / O request. Compared with other I / O technologies, it does not require maintenance of logical processing when data is not fully read, can reduce the number of threads, and thus can effectively cope with situations with large concurrency.
[0092] However, in the related art, there are mainly two methods for implementing asynchronous I / O. The first method is to implement asynchronous I / O by setting the file reading and writing mode to non-blocking mode. This method requires continuous query of data. When the I / O reading and writing are completed, the request initiator needs to be notified. The request initiator is responsible for the sending and receiving of data, and the processing efficiency is low. The second method is to implement asynchronous I / O through the select() interface, but each time the file descriptor array is input and retrieved, it involves copying the file descriptor array between the user state and the kernel state, and all the information in the file descriptor array needs to be traversed, which means that a lot of time and resources are needed for polling. In addition, in these two methods, the I / O request initiator also needs to process the return of I / O. Whether it is to obtain the return of I / O by notification or synchronous waiting, it will take up a certain amount of resource consumption, increase the request processing process, and is not suitable for scenarios with a large number of concurrency.
[0093] Based on this, an embodiment of the present application provides an asynchronous read-write method, which can be applied to an I / O interface. Based on the tasks contained in the read-write task request, target read-write tasks corresponding to the tasks are established respectively; the target read-write tasks are placed in the target read-write event; the target read-write tasks correspond to the target read-write events one by one; based on a triggering data set, the state of the target read-write event is determined; the triggering data set contains triggering data of the read-write event in the triggered state; when the state of the target read-write event is the triggered state, the target read-write task corresponding to the target read-write event is processed based on the target read-write event. In this way, when the target read-write event is triggered, the read-write task request can be processed directly according to the target read-write event, without returning the data in the target read-write event to the task initiator, that is, the I / O request initiator, and then processed by the task initiator, thereby improving processing efficiency and reducing resource consumption.
[0094] According to an embodiment of the present application, an embodiment of an asynchronous reading and writing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0095] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG1 is a flow chart of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG1 , the flow chart includes the following steps:
[0096] Step S101: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established.
[0097] In the embodiment of the present application, when a business demand arises, the task initiator generates a read / write task request corresponding to the business demand, obtains data related to the business demand through I / O, performs relevant processing, and realizes a response to the business. At this time, a corresponding target read / write task is established to map the read / write task request to the target read / write task, and the target read / write task completes the response to the read / write task request, thereby realizing a response to the business.
[0098] In an embodiment of the present application, a read / write task request may include multiple tasks, the read / write task request is split into multiple tasks, and then target read / write tasks corresponding to these tasks are established respectively, so that the target read / write tasks complete the response to the corresponding tasks. When each task included in the read / write task request completes the response, it can be determined that the read / write task request response is complete; and because the read / write task request is split into multiple tasks for response, the read / write task request does not need to continue to occupy the process to wait for the return of the processed data after it is initiated, thereby realizing the asynchrony of the read / write task request.
[0099] Step S102: putting the target read / write task into the target read / write event.
[0100] In the embodiment of the present application, the target read-write event is the read-write event corresponding to the target read-write task. The target read-write task corresponds to the target read-write event one-to-one. Each read-write task, or each task included in each read-write task request, corresponds to a read-write event, and the relevant data of the corresponding read-write task request will be written into the read-write event. In other words, the pending data related to the task read-write request will be written into the target read-write event, and the target read-write task will also be written into the storage space in the target read-write event where the pending data is stored. When the pending data is written into the target read-write event, the pending data and the target read-write task can be simultaneously obtained from the storage space corresponding to the target read-write event.
[0101] Step S103: determining the state of the target read / write event based on the trigger data set.
[0102] In an embodiment of the present application, the trigger data set includes trigger data of read and write events in a triggered state, and the trigger data set is used to determine the read and write events in a triggered state from multiple read and write events to be responded to, so as to respond to these read and write events in the triggered state.
[0103] In the embodiment of the present application, the state of the target read / write event includes a triggered state and a to-be-triggered state. When the state of the target read / write event is the triggered state, the to-be-processed data has been written into the target read / write event; when the state of the target read / write event is the to-be-triggered state, the to-be-processed data has not yet been written into the target read / write event.
[0104] Step S104 : When the state of the target read / write event is a trigger state, processing the target read / write task corresponding to the target read / write event based on the target read / write event.
[0105] In an embodiment of the present application, when the data to be processed is written into the target read-write event, the target read-write event will be triggered, that is, the state of the target read-write event will be switched to the triggered state. The triggered state indicates that the data in the target read-write event is ready and the response to the read-write task request can be completed. At this time, based on the target read-write event, the data to be processed and the target read-write task are obtained, and the data to be processed is processed by the target read-write task, thereby completing the response to the target read-write task.
[0106] In an optional embodiment, the method further includes: determining that the read / write task request response is complete when the target read / write task corresponding to each task in the read / write task request is processed. In other words, the read / write task request is split into multiple tasks, and the tasks included in the read / write task request are processed separately. When each task in the read / write task request is processed, the read / write task request response is determined to be complete, and the read / write task request no longer needs to occupy the process waiting for data to be returned and processed, thereby achieving an asynchronous response to the read / write task request.
[0107] In the related art, even if asynchronous I / O is used for data reading and writing, I / O is usually used to obtain data to the task initiator, and the task initiator then processes the data accordingly to complete the response to the business needs. After the task initiator initiates the read / write task request, the task initiator still needs to wait for the data to return, and then perform relevant processing based on the returned data, so as to complete the response to the read / write task request. It can be seen that the asynchronous I / O reading and writing in the related art cannot be completely asynchronous, that is, the initiation and response of the read / write task request cannot be completely separated. During the response process of the read / write task request, a certain amount of resources need to be occupied to wait for or obtain the return of data. Therefore, it is not suitable for scenarios with a large number of concurrency. In scenarios with a large number of concurrency, resource consumption will increase and processing efficiency will be reduced.
[0108] In the embodiment of the present application, the task read-write request is mapped to the target read-write task corresponding to the task it contains, and the response to the read-write task request is completed through the target read-write task, and the target read-write task is written into the corresponding target read-write event. When the target read-write event is triggered, the data to be processed and the target read-write task can be obtained synchronously. At this time, the data to be processed can be processed by the target read-write task, without the need to return the data to be processed to the task initiator and then processed by the task initiator. In other words, for the task initiator, it is only necessary to initiate the task read-write request. After the task response end writes the target read-write task into the target read-write event, the business of the task initiator is considered to be completed and no longer needs to occupy the thread; the detailed data processing and result return are executed by the task response end, and the task initiator does not need to wait for the read data to return, thereby achieving true asynchrony, that is, the initiation and response completion of the read-write task request are completely separated, thereby reducing resource consumption and improving processing efficiency in the case of a large number of concurrencies.
[0109] The asynchronous read and write method provided in the example of this application directly processes the read and write task request according to the target read and write event when the target read and write event is triggered, without returning the data in the target read and write event to the task initiator for further processing by the task initiator, thereby improving processing efficiency, reducing resource consumption, and being applicable to a large number of concurrent scenarios.
[0110] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG2 is a second flow diagram of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG2 , the flow includes the following steps:
[0111] Step S201: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established.
[0112] Optionally, step S201 includes:
[0113] Step S2011: Determine the task configuration information of each task in the read / write task request.
[0114] In the embodiment of the present application, the task configuration information specifies the relevant files corresponding to the task and the relevant data processing method corresponding to the task. In other words, the task configuration information may include task identification information and a task callback function.
[0115] In an embodiment of the present application, the task identification information indicates the file corresponding to the task, and optionally, it can be the file descriptor (fd) of the file. Since everything is a file in Linux, not only ordinary files, directories and other information are files, but other contents, such as processes, disks, devices, socket settings, etc., are all abstracted into files and operated in the form of files. The file descriptor is an index created by the kernel to efficiently manage opened files, which is used to refer to the opened file. Then, the file descriptor (fd) can be used to determine the file that each operation targets or calls.
[0116] In the embodiment of the present application, the task callback function is the operation performed by the task on its corresponding file. It can call the relevant data in the file in the form of a callback function to complete the relevant operations required by the task.
[0117] For example, when a business need to read a key arises, the key file is opened to obtain the file descriptor secret_key_fd of the key file, and the callback function after the key file is read is defined as secret_key_callback(). Then the task identification information corresponding to the read and write task request of this business need is the file descriptor secret_key_fd of the key file, and the task callback function is secret_key_callback().
[0118] Step S2012: Based on the task configuration information of each task, a target read and write task corresponding to each task is established.
[0119] In the embodiment of the present application, the task configuration information includes task identification information and a task callback function. Accordingly, based on the task configuration information of each task, the target read-write task corresponding to each task is established, which also includes: determining the task identification information of the task as the target identification information of the target read-write task corresponding to the task; and determining the task callback function of the task as the target callback function of the target read-write task corresponding to the task. In other words, the relevant information and operation mode representing the read-write task request are synchronized to each target read-write task, and the read-write task request is mapped to the target read-write task, thereby realizing the response and processing of the read-write task request through the target read-write task, and realizing the separation and decoupling of the initiation and response of the read-write task request through the target read-write task.
[0120] In an optional implementation, a target read / write task is established by designing a custom data structure io_task. Alternatively, the data structure io_task corresponding to the target read / write task can be defined in the following form:
[0121] typedef struct
[0122] {
[0123] void(*notify)(void*);
[0124] int fd;
[0125] unsigned int flags;
[0126] }io_task;
[0127] Where io_task is the target read / write task, typedef is a type definition, and struct is a structure type. Therefore, typedef struct is a structure definition, indicating that io_task is a structure. The content between typedef struct and io_task is the structure of the data structure io_task. Optionally, the data structure io_task corresponding to the target read / write task includes a callback function pointer *notify, a file descriptor fd, and event flags. The file descriptor fd is the target identification information; the callback function pointer *notify is the target callback function.
[0128] In an optional embodiment, the event identifier flags is a type restriction for the file corresponding to the data structure io_task, that is, the file corresponding to the target identification information fd, which limits the relevant operation mode or mechanism when opening or calling the corresponding file. It can be any one or a combination of EFD_CLOEXEC, EFD_NONBLOCK and EFD_SEMAPHORE. Among them, EFD_CLOEXEC is a closing flag during execution, that is, the file is closed after the corresponding file is opened and the process makes the relevant call; EFD_NONBLOCK indicates that any system call that opens the corresponding file and returns a file descriptor, that is, the target identification information, will not block the process, that is, it will return immediately if the target identification information cannot be obtained; EFD_SEMAPHORE can provide a mechanism similar to a semaphore, which is equivalent to the mechanism protection when reading content from the target identification information. For example, when performing a read operation under the semophore semantics, the counter value, that is, the count value, is reduced by 1 each time data is read.
[0129] For example, when a business need to read a key arises, the key file is opened, and the key file's file descriptor, secret_key_fd, is obtained. The callback function after reading the key file is defined as secret_key_callback(). Then, the task identification information corresponding to the read / write task request of this business need is the key file's file descriptor, secret_key_fd, and the task callback function is secret_key_callback(). Therefore, the data structure corresponding to the business need to read the key can be defined as secret_key_io_task. Accordingly, secret_key_io_task.fd is assigned to secret_key_fd, secret_key_io_task.notify is assigned to secret_key_callback, and secret_key_io_task.flags is assigned according to the setting requirements, thereby creating the data structure for the target read / write task. Among them, secret_key_io_task.fd indicates that the file descriptor fd comes from secret_key_io_task, secret_key_io_task.notify indicates that the function pointer notify of the callback function comes from secret_key_io_task, and secret_key_io_task.flags indicates that the event identifier flags comes from secret_key_io_task.
[0130] Step S202: Place the target read / write task into the target read / write event. For details, please refer to step S102 of the embodiment shown in FIG1 , which will not be described in detail here.
[0131] Step S203: Determine the state of the target read / write event based on the trigger data set. Please refer to step S103 of the embodiment shown in FIG1 for details, which will not be repeated here.
[0132] Step S204: When the target read / write event is in the triggered state, the target read / write task corresponding to the target read / write event is processed based on the target read / write event.
[0133] The asynchronous read-write method provided in the embodiment of the present application maps the read-write task request to the target read-write task, and responds to and processes the read-write task request through the target read-write task, thereby achieving separation and decoupling of the initiation and response of the read-write task request.
[0134] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG3 is a flow diagram of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG3 , the flow includes the following steps:
[0135] Step S301: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established. For details, please refer to step S101 of the embodiment shown in FIG1 , which will not be described in detail here.
[0136] Step S302: Put the target read / write task into the target read / write event.
[0137] Optionally, step S302 includes:
[0138] Step S3021: Determine the target read / write event corresponding to the target read / write task.
[0139] In the embodiment of the present application, a target read / write event may include three parts: event identification information, event type information, and event data. The event identification information is consistent with the target identification information of the target read / write task corresponding to the target read / write event, thereby associating the target read / write event with the target read / write task through the event identification information; the event type information is the event type of the target read / write event, indicating the operation type or trigger type of the target read / write event; and the event data is used to store the data to be processed.
[0140] In an optional implementation, step S3021 may include:
[0141] Step a1: Use the target identification information of the target read / write task as the event identification information of the target read / write event.
[0142] In an embodiment of the present application, the target identification information of the target read-write task is used as the event identification information of the target read-write event to establish a correspondence between the target read-write task and the target read-write event, so that the target read-write event corresponding to the target read-write task can be determined through the target identification information, and the target read-write task corresponding to the target read-write event can be determined through the event identification information.
[0143] Step a2: Determine the event type information of the target read / write event.
[0144] In the embodiment of the present application, the event type information may include the following types of content, which can be set according to the needs of the read / write task request:
[0145] 1.EPOLLIN: Indicates that the monitored file has readable data;
[0146] 2. EPOLLOUT: Indicates that the monitored file has data that can be written;
[0147] 3. EPOLLPRI: Indicates that the monitored file has readable high-priority data;
[0148] 4.EPOLLERR: indicates that an exception occurs in the corresponding file;
[0149] 5.EPOLLHUP: indicates that the corresponding file is suspended;
[0150] 6.EPOLLET: indicates the use of edge triggering;
[0151] 7.EPOLLONESHOT: Indicates that after an event is generated and processed, the file is no longer monitored.
[0152] In this way, the target read / write task and the target read / write event can be matched through the target identification information of the target read / write task and the event identification information of the target read / write event.
[0153] In an optional embodiment, since the task response end queries and calls data based on the task instance, in order to enable the task response end to confirm the data update status in the target read / write event in real time, it is also necessary to associate the target read / write task with the task instance, thereby adding the target read / write event to the event set of the task instance. Optionally, the method further includes: associating the task instance and the target read / write task based on the task control function, so as to add the target read / write event to the event set of the task instance.
[0154] In an optional embodiment, based on a task control function, a task instance and a target read-write task are associated to add a target read-write event to an event collection of the task instance, including: setting the instance identifier in the task control function to the instance identifier information of the task instance; setting the task identifier in the task control function to the target identifier information of the target read-write task; setting the operation parameter in the task control function to a value corresponding to the added event; calling the task control function to associate the task instance and the target read-write task, and adding the target read-write event to the event collection of the task instance.
[0155] In an optional implementation, the target read / write task can be associated with the task instance using the epoll interface. The epoll interface was developed to address the Linux kernel's difficulty handling large numbers of file descriptors. It enhances the select and poll interfaces used in multiplexed I / O in Linux. The epoll interface can significantly improve processor utilization in high-concurrency service programs, particularly when only a small number of connections are active among a large number of concurrent connections.
[0156] In an optional embodiment, the task control function can be the event registration function epoll_ctl() function of the epoll interface. A successful call to the epoll_ctl() function will associate the task instance with the target read-write task, and it can control and modify the events on each read-write task in the task instance. The epoll_ctl() function is constructed as follows: int epoll_ctl(int epfd, int op, int fd, struct epoll_event*event). Among them, epfd is the instance identifier, which is a file descriptor dedicated to epoll. This file descriptor actually has nothing to do with the real file and is only used for related calls under the subsequent epoll mechanism; op is the operation parameter, which is the operation on the file corresponding to the file descriptor fd; fd is the task identifier, that is, the file descriptor, corresponding to the target identification information; epoll_event is a structure about the target read-write event, and *event is a pointer to a target read-write event. By assigning values to the various parameters in the epoll_ctl() function, the instance identifier is assigned to the instance identifier information of the task instance, the operation parameter is assigned to the value corresponding to the added event, and the task identifier is assigned to the target identifier information of the target read-write task, that is, the file descriptor corresponding to the target read-write task. Thus, through the task control function, the task instance and the target read-write task are associated, so that the target read-write event is added to the event collection of the task instance.
[0157] In an optional implementation, by setting the operation parameter op to different values, the task control function can adjust the read and write events contained in the event set of the task instance, as well as the related information of the read and write events. The operation parameter op can include values corresponding to add events, modify events, and delete events, respectively. Optionally, it can be as follows:
[0158] EPOLL_CTL_ADD: Add an event. Add the file corresponding to the file descriptor fd to the epoll instance listener set corresponding to the instance identifier epfd, and listen for events defined by the epoll_event structure. In other words, associate the target read / write task with the task instance, add the file corresponding to the target read / write task to the task instance's listener set, and add the target read / write event to the task instance's event collection.
[0159] EPOLL_CTL_DEL: Delete event. Delete the file corresponding to the file descriptor fd from the epoll instance monitor set corresponding to the instance identifier epfd; that is, cancel the association between the target read / write task and the task instance, remove the file corresponding to the target read / write task from the task instance monitor set, and remove the target read / write event from the task instance event set.
[0160] EPOLL_CTL_MOD: Modify events. Modify the target read / write event information in the event collection of the task instance, changing the monitoring behavior of the file corresponding to the target read / write task.
[0161] In an optional implementation, a task instance can be created through the creation function epoll_create() of the epoll interface, and the file descriptor of the task instance, that is, the instance identifier epfd, can be obtained. Optionally, the epoll_create() function is constructed as follows: int epoll_create(int size). Among them, size is the maximum number of files that can be monitored on the task instance, that is, the maximum number of target read and write tasks that can be associated with the task instance. When the call to the epoll_create() function is successful, the epoll_create() function will create an epoll task instance and return the instance identifier epfd; when the call to the epoll_create() function fails, the epoll_create() function returns -1 and sets the error code errno according to the cause of the error. Optionally, the error code may include: EINVAL, indicating that size is not a positive number; ENFILE, indicating that the number of files opened by the system has reached the upper limit; ENOMEN, indicating that there is not enough memory to complete the operation.
[0162] Step S3022: Write the target read / write task into the event data of the target read / write event.
[0163] In an embodiment of the present application, as explained above, the target read and write event may include three parts, namely event identification information, event type information and event data, wherein the event identification information is consistent with the target identification information of the target read and write task corresponding to the target read and write event, thereby corresponding the target read and write event to the target read and write task through the event identification information; the event type information is the event type of the target read and write event, indicating the operation type or trigger type of the target read and write event; the event data is used to store the data to be processed.
[0164] In an optional implementation, in an epoll system, the target read and write event may be in the form of an epoll_event structure. Optionally, the information contained in the epoll_event structure is as follows:
[0165] The events field contains event type information, and its value is as explained in step S3021 above. The data field represents user data, and its type is union, that is, a union type, which can store data such as a pointer or file descriptor. It is defined as follows:
[0166] Here, ptr can point to any type of user data, corresponding to event data; fd represents a file descriptor, corresponding to event identification information; u32 and u64 represent 32-bit and 64-bit unsigned integers, respectively. In actual use, users can store the required data (the data to be processed) in this field and in the ptr of the data field, that is, ptr points to the data to be processed. When the target read or write event is triggered, the system will call and return the data to be processed pointed to by ptr to process the target read or write event.
[0167] As can be seen from the above, in the epoll system, the event data of the target read / write event is stored in epoll_event.data.ptr, that is, in the data field of the epoll_event structure. Then, the target read / write task is written into the event data of the target read / write event, that is, io_task is assigned to epoll_event.data.ptr, making the target read / write task io_task a part of the event data epoll_event.data.ptr of the target read / write event epoll_event. Therefore, when the target read-write event epoll_event is triggered and the event data epoll_event.data.ptr of the target read-write event epoll_event is obtained, not only the data to be processed stored in the event data epoll_event.data.ptr can be obtained, but also the target read-write task io_task can be obtained. The target read-write task io_task is a mapping of the read-write task request, which reflects the processing requirements and operations of the read-write task request on the data to be processed in the target read-write event. Then, the data to be processed can be processed according to the target read-write task io_task, thereby completing the response and processing of the read-write task request. Therefore, after the separation and decoupling of the initiation and response of the read-write task request, the response to the read-write task request can still be completed in a timely manner.
[0168] Step S303: Determine the state of the target read / write event based on the trigger data set. Please refer to step S103 of the embodiment shown in FIG1 for details, which will not be repeated here.
[0169] Step S304: When the target read / write event is in the trigger state, the target read / write task corresponding to the target read / write event is processed based on the target read / write event.
[0170] The asynchronous read-write method provided in the embodiment of the present application is designed based on the epoll system on the one hand, and can make full use of the efficient concurrent nature of the epoll system to improve processing efficiency and reduce resource consumption. On the other hand, by writing the target read-write task into the event data of the target read-write event, the target read-write task is used as part of the event data of the target read-write event. When the target read-write event is triggered and the event data of the target read-write event is obtained, the target read-write task and the data to be processed are obtained at the same time. The data to be processed is processed by the target read-write task, so that after the initiation and response of the read-write task request are separated and decoupled, the response to the read-write task request can still be completed in a timely manner.
[0171] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG4 is a fourth flow chart of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG4 , the flow chart includes the following steps:
[0172] Step S401: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established. For details, please refer to step S101 of the embodiment shown in FIG1 , which will not be described in detail here.
[0173] Step S402: Place the target read / write task into the target read / write event. For details, please refer to step S102 of the embodiment shown in FIG1 , which will not be described in detail here.
[0174] Step S403: determining the status of the target read / write event based on the trigger data set.
[0175] Optionally, step S403 includes:
[0176] Step S4031: Acquire a trigger data set.
[0177] In the embodiment of the present application, the trigger data set is used to determine the read / write events in the triggered state from the event set of the task instance. When the state of the target read / write event in the event set of the task instance is the triggered state, the relevant information of the target read / write event is placed into the trigger data set through the waiting function epoll_wait(), and then the trigger data set is returned. Based on the trigger data set, it is possible to determine which target read / write events have updated pending data, and obtain the pending data from the trigger data set.
[0178] In an optional embodiment, the method further includes the steps of modifying the state of the target read / write event for the data to be processed to be updated to a triggered state, and adding the target read / write event to the trigger data set. Optionally, modifying the state of the target read / write event for the data to be processed to be updated to a triggered state, and adding the target read / write event to the trigger data set may include: when the data to be processed is written into the event data of the target read / write event, using the event type information and event identification information of the target read / write event as the trigger data of the target read / write event; and adding the trigger data of the target read / write event to the trigger data set.
[0179] In an optional embodiment, the event identification information, that is, the file descriptor fd, can uniquely determine the target read / write event and the file opened and operated by the corresponding target read / write task; and for the same file, different operation types may be generated. Business needs, that is, target read / write events and target read / write tasks corresponding to different operation types, then the event type information can determine the operation type of the target read / write event and target read / write task corresponding to the file. In other words, based on the event identification information and event type information, a target read / write event can be uniquely determined, and the operation type of the target read / write event can be determined. For example, the event set of the task instance includes a read event for a key file, the event identification information of the read event for the key file is secret_key_fd, and the event type information of the read event for the key file is EPOLLIN. Then, when there is a trigger data with event identification information secret_key_fd and event type information EPOLLIN in the trigger data set, it can be determined that the target read / write event corresponding to the trigger data is the read event for the key file. Therefore, it is necessary to add the event identification information and event type information as the trigger data of the target read / write event to the trigger data set, so as to uniquely determine the file corresponding to the target read / write event and the type of operation to be performed. Thus, when the data to be processed is written to the target read / write event, the target read / write event is triggered by constructing the trigger data and writing the trigger data into the trigger data set.
[0180] In an optional embodiment, after adding the trigger data of the target read / write event to the trigger data set, a step is also included to clear the event type information in the target read / write event to indicate that the target read / write event has been triggered and its corresponding read / write task request has generated a response.
[0181] Step S4032: When the event identification information in the trigger data set is consistent with the target identification information of the target read / write task, the state of the target read / write event is determined to be a trigger state.
[0182] As described above, when the pending data is written into the event data of the target read / write event, the target read / write event is triggered. The waiting function epoll_wait() adds the target read / write event to the trigger data set and then returns the trigger data set. After returning the trigger data set, it is possible to determine which target read / write events are in the triggered state based on the trigger data set. After determining that the state of the target read / write event is the triggered state, it is possible to determine that the pending data has been written into the event data of the target read / write event. At this time, it is possible to complete the response to the read / write task request based on the target read / write event. Therefore, it is possible to determine from the trigger data set whether the read / write events in the triggered state contain the target read / write event based on the event identification information.
[0183] Step S404: When the target read / write event is in the trigger state, the target read / write task corresponding to the target read / write event is processed based on the target read / write event.
[0184] The asynchronous read and write method provided in the embodiment of the present application forms trigger data by forming event identification information and event type information of the read and write events in the triggered state, and putting the information into a trigger data set. By searching the trigger data set, the read and write events in the triggered state can be determined, thereby completing the processing of the read and write tasks corresponding to the read and write events.
[0185] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG5 is a flowchart of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG5 , the flowchart includes the following steps:
[0186] Step S501: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established. For details, please refer to step S101 of the embodiment shown in FIG1 , which will not be described in detail here.
[0187] Step S502: Place the target read / write task into the target read / write event. For details, please refer to step S102 of the embodiment shown in FIG1 , which will not be described in detail here.
[0188] Step S503: Determine the state of the target read / write event based on the trigger data set. Please refer to step S103 of the embodiment shown in FIG1 for details, which will not be repeated here.
[0189] Step S504 : When the state of the target read / write event is the trigger state, the target read / write task corresponding to the target read / write event is processed based on the target read / write event.
[0190] Optionally, step S504 includes:
[0191] Step S5041: Acquire event data of the target read / write event.
[0192] In the embodiment of the present application, the event data of the target read-write event can be obtained by the wait function epoll_wait() function of the epoll interface. By cyclically calling the wait function epoll_wait() function, it is determined whether the data of the target read-write event contained in the task instance is updated. When the data of the target read-write event is updated, the target read-write event is triggered, that is, the state of the target read-write event is in the triggered state. At this time, the wait function epoll_wait() function returns the relevant data of the target read-write event in the triggered state. The wait function epoll_wait() function is constructed as follows:
[0193] int epoll_wait(int epfd, struct epoll_event*events, int maxevents, int timeout)
[0194] Among them, epfd is the instance identifier, which is a file descriptor dedicated to epoll; epoll_event is a structure about target read and write events, and *event is a pointer to a target read and write event; maxevents is the maximum number of members contained in the trigger event array. The trigger event array is an array composed of target read and write events in the triggered state in the task instance in the current call. The number of members of the array is also the number of target read and write events in the triggered state in the current call; timeout is the timeout period. When no target read and write event is triggered after the timeout period, the waiting function epoll_wait() returns 0, indicating that it has timed out. In particular, when timeout is set to 0, even if no target read and write event is triggered, the call occurs immediately. At this time, the waiting function epoll_wait() returns 0. When timeout is set to -1, the call waits until an event occurs.
[0195] Step S5042: extract the target read / write task and the data to be processed from the event data.
[0196] In the embodiment of the present application, the target read and write task and the data to be processed are taken out from the event data, that is, the io_task and the data to be processed are taken out from epoll_event.data.ptr.
[0197] Step S5043: Process the data to be processed based on the target read and write task.
[0198] In the embodiment of the present application, since the target read / write task is a mapping to the read / write task request, based on the target read / write task, the data to be processed can be processed to complete the response to the read / write task request.
[0199] In an optional implementation, step S5043 may include:
[0200] Step b1, obtaining the target callback function in the target read / write task.
[0201] Step b2: call the target callback function to process the data to be processed.
[0202] In the embodiment of the present application, the target read / write task io_task includes a target callback function *notify. The target callback function *notify is a mapping of the task callback function to the read / write task request. The processing of the pending data based on the target callback function *notify can complete the response to the read / write task request. Thus, according to the target callback function corresponding to the read / write task request, the pending data is processed, thereby completing the data processing according to the task requirements corresponding to the target read / write task.
[0203] In an optional implementation, when the task waiting function returns a first preset value, ie, -1, it is determined that no data is written in the target read / write event within the timeout period, and the read / write task request is not responded to.
[0204] The asynchronous read and write method provided in the embodiment of the present application can process the data to be processed through the target read and write task contained in the event data when the target read and write event is triggered. There is no need to take out the data to be processed and return it to the task initiator, thereby realizing the separation and decoupling of the initiation and response of the read and write task request, and improving processing efficiency.
[0205] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG6 is a sixth flow chart of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG6 , the flow chart includes the following steps:
[0206] Step S601: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established. For details, please refer to step S101 of the embodiment shown in FIG1 , which will not be described in detail here.
[0207] Step S602: Place the target read / write task into the target read / write event. For details, please refer to step S102 of the embodiment shown in FIG1 , which will not be described in detail here.
[0208] Step S603: Determine the state of the target read / write event based on the trigger data set. Please refer to step S103 of the embodiment shown in FIG1 for details, which will not be repeated here.
[0209] Step S604: When the target read / write event is in the trigger state, the target read / write task corresponding to the target read / write event is processed based on the target read / write event.
[0210] Step S605: When the number of responses to the read / write task request reaches a preset number of responses, the association between the task instance and the target read / write task is released.
[0211] In an embodiment of the present application, for read / write task requests that require multiple responses or are non-temporary responses, a preset number of responses can be set. The preset number of responses can be set based on the frequency of occurrence of the read / write task request in the system. Within the preset number of responses, each time a response to the read / write task request is completed, the association between the task instance and the target read / write task remains. When the number of responses to the read / write task request reaches the preset number of responses, the association between the task instance and the target read / write task is released.
[0212] In an optional implementation, the preset number of responses corresponding to temporary responses, or responses to read and write task requests that occur less frequently than a certain frequency, can be set to 1. After the response is completed, the association between the task instance and its corresponding target read and write task is directly contacted to avoid occupying resources in the task instance.
[0213] In an optional embodiment, a preset number of responses for related read and write task requests can be pre-set in the system. For read and write task requests that do not have a preset number of responses, the preset number of responses is uniformly defaulted to 1, that is, the association between the task instance and the target read and write task is released after completing one response.
[0214] Optionally, step S605 includes:
[0215] Step S6051: Set the operation parameter in the task control function to the value corresponding to the deletion event.
[0216] In the embodiment of the present application, as explained in the above step S3021, the operation parameter op in the task control function epoll_ctl() function is set to the value EPOLL_CTL_DEL corresponding to the deletion event.
[0217] Step S6052: calling the task control function to disassociate the task instance from the target read / write task, and deleting the target read / write event from the event set of the task instance.
[0218] In an embodiment of the present application, the operation parameter op is set to the value EPOLL_CTL_DEL corresponding to the deletion event, the file descriptor fd is set to the target identification information of the target read and write task, and then the task control function epoll_ctl() function is called to disassociate the task instance and the target read and write task, and delete the target read and write event from the event set of the task instance.
[0219] In this embodiment, an asynchronous read / write method is provided, which can be used for the above-mentioned I / O interface. FIG7 is a flow chart of an asynchronous read / write method according to an embodiment of the present application. As shown in FIG7 , the flow chart includes the following steps:
[0220] Step S701: Based on the tasks included in the read / write task request, target read / write tasks corresponding to the tasks are respectively established. For details, please refer to step S101 of the embodiment shown in FIG1 , which will not be described in detail here.
[0221] Step S702: Place the target read / write task into the target read / write event. For details, please refer to step S102 of the embodiment shown in FIG1 , which will not be described in detail here.
[0222] Step S703: Determine the state of the target read / write event based on the trigger data set. Please refer to step S103 of the embodiment shown in FIG1 for details, which will not be repeated here.
[0223] Step S704: When the target read / write event is in the trigger state, the target read / write task corresponding to the target read / write event is processed based on the target read / write event.
[0224] Step S705: When the number of responses to the read / write task request does not reach the preset number of responses, the target read / write event is modified to supplement the event type information in the target read / write event.
[0225] In an embodiment of the present application, for read-write task requests with multiple response requirements or non-temporary responses, a preset number of responses can be set, and the preset number of responses can be set according to the frequency of occurrence of the read-write task request in the system. Within the preset number of responses, each time the response to the read-write task request is completed, the association between the task instance and the target read-write task will not be released. In addition, since the event type information of the target read-write event is cleared when the trigger data of the target read-write event is added to the trigger data set, it is necessary to re-supplement the event type information in the target read-write event to ensure that the target read-write event can be called normally.
[0226] Optionally, step S705 includes:
[0227] Step S7051: Determine the event type information of the target read / write event.
[0228] In the embodiment of the present application, the event type information of the target read / write event is re-determined according to the requirements of the task read / write request, and the event type information set each time can be different to meet different requirements.
[0229] Step S7052: Set the operation parameter in the task control function to the value corresponding to the modification event.
[0230] In the embodiment of the present application, as in the above step S3021, the operation parameter op in the task control function epoll_ctl() function is set to the value EPOLL_CTL_MOD corresponding to the modification event.
[0231] Step S7053: calling the task control function to modify the target read / write event to supplement the event type information in the target read / write event.
[0232] In an embodiment of the present application, the operation parameter op is set to the value EPOLL_CTL_MOD corresponding to the modification event, the file descriptor fd is set to the target identification information of the target read and write task, and then the task control function epoll_ctl() function is called to modify the target read and write event to supplement the event type information in the target read and write.
[0233] The asynchronous reading and writing method provided in the embodiment of the present application pre-sets a preset number of responses. When the number of responses to the reading and writing task request reaches the required preset number of responses, the association between the task instance and the target reading and writing task is disassociated, thereby avoiding repeatedly establishing and disassociating the task instance and the target reading and writing task, thereby improving processing efficiency.
[0234] Figure 8 is a schematic diagram of the thread flow of the asynchronous read / write method provided in this example. The following, in conjunction with Figure 8, details the decoupling of the initiation and response of read / write task requests in this method. As shown in Figure 8, during the execution of the loop business task (loop business_task), business_thread is the business initiation thread, configured to initiate read / write task requests, and epoll_thread is the business response thread, configured to respond to read / write task requests based on the epoll mechanism. The response processing method may include, but is not limited to, loop polling (long polling). After initializing and creating the target read-write task io_task, the business initiation thread business_thread establishes an association between the target read-write task io_task and the target read-write event epoll_event, that is, writing the target read-write task io_task into the event data in the target read-write event epoll_event, and establishing an association between the target read-write task io_task and the task instance by calling the task control function epoll_ctl() function. At this point, the business initiation thread epoll_thread completes the processing; the business response thread epoll_thread calls the epoll_wait() function in a loop. When the target read-write event epoll_event is triggered, it obtains the event data of the target read-write event epoll_event, and takes out the target read-write task io_task and the data to be processed from it, and calls the target callback function notify in the target read-write task io_task to process the data to be processed. At this point, the business response thread epoll_thread completes the processing. The business initiation thread business_thread and the business response thread epoll_thread are completely independent, separating and decoupling the initiation and response of read and write task requests, thereby realizing asynchronous response to read and write task requests.
[0235] In the present embodiment, an asynchronous read / write device is also provided, which is configured to implement the above-mentioned embodiment and optional implementation methods. The details already described will not be repeated here. As used below, the terms "module" and "unit" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.
[0236] This embodiment provides an asynchronous read / write device, as shown in FIG9 , including:
[0237] The target reading and writing task establishing module 901 is configured to establish target reading and writing tasks corresponding to the tasks respectively based on the tasks included in the reading and writing task request;
[0238] The target read / write event writing module 902 is configured to put the target read / write task into the target read / write event; the target read / write task corresponds to the target read / write event in a one-to-one manner;
[0239] The read / write event state determination module 903 is configured to determine the state of the target read / write event based on a trigger data set; the trigger data set includes trigger data of the read / write event in the triggered state;
[0240] The read / write task request processing module 904 is configured to process a target read / write task corresponding to the target read / write event based on the target read / write event when the state of the target read / write event is a trigger state.
[0241] In an optional implementation, the target reading and writing task establishing module 901 includes:
[0242] a task configuration information determining unit configured to determine task configuration information of a read / write task request;
[0243] The target reading and writing task establishing unit is configured to establish the target reading and writing task based on the task configuration information.
[0244] In an optional implementation, the task configuration information determining unit includes:
[0245] The target identification information determining subunit is configured to determine the task identification information of the task as the target identification information of the target reading and writing task corresponding to the task;
[0246] The target callback function determination subunit is configured to determine the task callback function of the task as the target callback function of the target read / write task corresponding to the task.
[0247] In an optional implementation, the target read / write event writing module 902 includes:
[0248] a target read / write event determining unit, configured to determine a target read / write event corresponding to a target read / write task;
[0249] The target read / write task writing unit is configured to write the target read / write task into the event data of the target read / write event.
[0250] In an optional implementation, the target read / write event determination unit includes:
[0251] The event identification information determining subunit is configured to use the target identification information of the target read / write task as the event identification information of the target read / write event;
[0252] The event type information determination subunit is configured to determine the event type information of the target read / write event.
[0253] In an optional embodiment, the device further includes:
[0254] The task instance associating device is configured to associate the task instance with the target read / write task based on the task control function, so as to add the target read / write event to the event set of the task instance.
[0255] In an optional embodiment, the task instance associating device includes:
[0256] an instance identification determination unit configured to set the instance identification in the task control function to instance identification information of the task instance;
[0257] A task identification determination unit is configured to set the task identification in the task control function as target identification information of the target read / write task;
[0258] an operation parameter determination unit configured to set an operation parameter in a task control function to a value corresponding to the added event;
[0259] The task control function calling unit is configured to call the task control function to associate the task instance with the target read / write task and add the target read / write event to the event set of the task instance.
[0260] In an optional implementation, the read / write event status determination module 903 includes:
[0261] a trigger data set acquisition unit, configured to acquire a trigger data set;
[0262] The trigger state determining unit is configured to determine that the state of the target read / write event is a trigger state when event identification information in the trigger data set is consistent with target identification information of the target read / write task.
[0263] In an optional embodiment, the device further includes:
[0264] The trigger data determination module is configured to use the event type information and event identification information of the target read / write event as the trigger data of the target read / write event when the to-be-processed data is written into the event data of the target read / write event;
[0265] The trigger data set determination module is configured to add the trigger data of the target read / write event into the trigger data set.
[0266] In an optional embodiment, the device further includes:
[0267] The event type information clearing module is configured to clear the event type information in the target read / write event.
[0268] In an optional implementation, the read / write task request processing module 904 includes:
[0269] an event data acquisition unit configured to acquire event data of a target read / write event;
[0270] An event data separation unit is configured to extract target read and write tasks and data to be processed from event data;
[0271] The data processing unit to be processed is configured to process the data to be processed based on the target reading and writing tasks.
[0272] In an optional embodiment, the data processing unit to be processed includes:
[0273] The target callback function acquisition subunit is set to acquire the target callback function in the target read and write task;
[0274] The data-to-be-processed processing subunit is configured to call a target callback function to process the data-to-be-processed.
[0275] In an optional embodiment, the device further includes:
[0276] The association removal module is configured to remove the association between the task instance and the target read / write task when the number of responses to the read / write task request reaches a preset number of responses.
[0277] In an optional implementation, the association removal module includes:
[0278] an operation parameter setting unit configured to set an operation parameter in a task control function to a value corresponding to a deletion event;
[0279] The association removal unit is configured to call a task control function to remove the association between the task instance and the target read / write task, and delete the target read / write event from the event set of the task instance.
[0280] In an optional embodiment, the device further includes:
[0281] The event type information supplement module is configured to modify the target read / write event to supplement the event type information in the target read / write event when the number of responses to the read / write task request does not reach a preset number of responses.
[0282] In an optional implementation, the event type information supplementing module includes:
[0283] an event type information determining unit, configured to determine event type information of a target read / write event;
[0284] an operation parameter setting unit configured to set an operation parameter in a task control function to a value corresponding to a modification event;
[0285] The event type information supplement unit is configured to call a task control function to modify a target read / write event to supplement the event type information in the target read / write event.
[0286] In an optional embodiment, the device further includes:
[0287] The task request unresponse module is configured to determine that the read / write task request is unresponded when the task wait function returns a first preset value.
[0288] In an optional embodiment, the device further includes:
[0289] The task request response module is configured to determine that the read / write task request response is completed when the target read / write task corresponding to each task in the read / write task request is processed completely.
[0290] The more detailed functional descriptions of the above modules and units are the same as those in the above corresponding embodiments and will not be repeated here.
[0291] The asynchronous read / write device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0292] An embodiment of the present application also provides a computer device having the asynchronous reading and writing device shown in FIG. 9 above.
[0293] Please refer to Figure 10, which is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present application. As shown in Figure 10, the computer device includes: one or more processors 10, a memory 20, and interfaces configured to connect various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, each device providing some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 10 takes a processor 10 as an example.
[0294] The processor 10 may be a central processing unit (CPU), a network processor (NPU), or a combination thereof. The processor 10 may also include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device (PLD) may be a complex programmable logic device (CPLD), a field programmable gate array (FPGA), a general purpose array logic (GAL), or any combination thereof.
[0295] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.
[0296] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0297] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.
[0298] The computer device further comprises a communication interface 30, which is configured to communicate with other devices or a communication network.
[0299] The embodiments of the present application also provide a non-volatile readable storage medium. The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a non-volatile readable storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; optionally, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.
[0300] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the embodiments of the present application, and such modifications and variations shall fall within the scope defined by the appended claims.
Claims
1. An asynchronous reading and writing method, characterized in that: The method comprises: Based on the tasks included in the read and write task requests, target read and write tasks corresponding to the tasks are respectively established; Put the target read-write task into the target read-write event; the target read-write task corresponds to the target read-write event one by one; Determining the state of the target read / write event based on a trigger data set; the trigger data set includes trigger data of the read / write event in a triggered state; When the state of the target read-write event is a trigger state, based on the target read-write event, the target read-write task corresponding to the target read-write event is processed.
2. The method according to claim 1, characterized in that The tasks included in the read / write task request are respectively based on the tasks corresponding to the target read / write tasks, including: Determine task configuration information for each of the tasks in the read / write task request; Based on the task configuration information of each task, the target reading and writing task corresponding to each task is established respectively.
3. The method according to claim 2, characterized in that Based on the task configuration information of each task, the target read / write task corresponding to each task is established respectively, including: Determine the task identification information of the task as the target identification information of the target read-write task corresponding to the task; The task callback function of the task is determined as the target callback function of the target read-write task corresponding to the task.
4. The method according to claim 1, characterized in that: The step of placing the target read-write task into a target read-write event includes: Determine the target read-write event corresponding to the target read-write task; The target read / write task is written into the event data of the target read / write event.
5. The method according to claim 4, characterized in that The determining the target read / write event corresponding to the target read / write task includes: Using the target identification information of the target read-write task as the event identification information of the target read-write event; Determine the event type information of the target read / write event.
6. The method according to claim 1, characterized in that The method further comprises: Based on the task control function, the task instance and the target read-write task are associated to add the target read-write event to the event set of the task instance.
7. The method according to claim 6, characterized in that The step of associating the task instance and the target read / write task based on the task control function to add the target read / write event to the event set of the task instance includes: Setting the instance identifier in the task control function to the instance identifier information of the task instance; Setting the task identifier in the task control function to the target identifier information of the target read-write task; Setting the operation parameter in the task control function to a value corresponding to the added event; The task control function is called to associate the task instance with the target read-write task, and to add the target read-write event to the event set of the task instance.
8. The method according to claim 1, characterized in that The determining the state of the target read / write event based on the trigger data set includes: Acquiring the trigger data set; When the event identification information in the trigger data set is consistent with the target identification information of the target read-write task, it is determined that the state of the target read-write event is the trigger state.
9. The method according to claim 1, characterized in that: The method further comprises: When the data to be processed is written into the event data of the target read / write event, the event type information and event identification information of the target read / write event are used as the trigger data of the target read / write event; The trigger data of the target read / write event is added to the trigger data set.
10. The method according to claim 9, characterized in that After adding the trigger data of the target read / write event to the trigger data set, the method further includes: Clear the event type information in the target read / write event.
11. The method according to claim 1, characterized in that: The processing of the read / write task request corresponding to the target read / write event based on the target read / write event includes: Acquire event data of the target read / write event; Taking out the target read / write task and the data to be processed from the event data; Based on the target reading and writing task, the data to be processed is processed.
12. The method according to claim 11, characterized in that The processing of the data to be processed based on the target reading and writing task includes: Obtain the target callback function in the target read-write task; The target callback function is called to process the data to be processed.
13. The method according to any one of claims 1 to 12, characterized in that: After processing the read / write task request based on the target read / write event, the method further includes: When the number of responses to the read / write task request reaches a preset number of responses, the association between the task instance and the target read / write task is released.
14. The method according to claim 13, characterized in that The removing the association between the task instance and the target read-write task includes: Set the operation parameter in the task control function to the value corresponding to the deletion event; The task control function is called to disassociate the task instance from the target read-write task, and to delete the target read-write event from the event set of the task instance.
15. The method according to any one of claims 1 to 12, characterized in that: After processing the read / write task request based on the target read / write event, the method further includes: When the number of responses to the read / write task request does not reach the preset number of responses, the target read / write event is modified to supplement the event type information in the target read / write event.
16. The method according to claim 14, characterized in that The modifying the target read / write event to supplement the event type information in the target read / write event includes: Determine event type information of the target read / write event; Set the operation parameter in the task control function to the value corresponding to the modification event; The task control function is called to modify the target read / write event to supplement the event type information in the target read / write event.
17. The method according to claim 1, characterized in that The method further comprises: When the task waiting function returns the first preset value, it is determined that the read / write task request is not responded to.
18. The method according to claim 1, characterized in that The method further comprises: When the target read / write task corresponding to each task in the read / write task request is processed completely, it is determined that the read / write task request response is completed.
19. An asynchronous read-write device, characterized in that: The device comprises: A target reading and writing task establishing module is configured to establish target reading and writing tasks corresponding to the tasks respectively based on the tasks included in the reading and writing task request; A target read-write event writing module is configured to put the target read-write task into a target read-write event; the target read-write task corresponds to the target read-write event one by one; The read / write event state determination module is configured to determine the state of the target read / write event based on a trigger data set; the trigger data set Contains trigger data of read and write events in the triggered state; The read / write task request processing module is configured to process the target read / write task corresponding to the target read / write event based on the target read / write event when the state of the target read / write event is a trigger state.
20. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the asynchronous reading and writing method according to any one of claims 1 to 18 by executing the computer instructions.
21. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores computer instructions, and the computer instructions are configured to enable a computer to execute the asynchronous reading and writing method according to any one of claims 1 to 18.
Citation Information
Patent Citations
Client and asynchronous processing method for client based on Memcache protocol
CN106815083A
Event processing method and device, apparatus, and readable storage medium
CN109002363A
Asynchronous read-write method and device, computer equipment and storage medium
CN116820430A
Method for prioritized event processing in an event dispatching system
US20090288089A1