Network data processing method and device, server and storage medium

Abstract

The network data processing method and device, server and storage medium provided by the present application relate to the field of data processing technology. In the present application, first, a data receiving coroutine is used to detect whether a target data packet exists. Secondly, if a target data packet exists, a corresponding network coroutine is determined in multiple network coroutines through the data receiving coroutine, and the network coroutine is placed in a pre-configured network linked list. Then, if there is no target data packet, after switching the data receiving coroutine to the main coroutine of the target network thread, the main coroutine is switched to each network coroutine in the network linked list in turn, so as to process the corresponding target data packet through the network coroutine, and after each network coroutine in the network linked list is switched, the main coroutine is switched to the data receiving coroutine to detect the task queue again. Through the above method, the problem of low data processing efficiency existing in the existing data processing technology can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a network data processing method and device, a server, and a storage medium. Background Art

[0002] In traditional data processing technology, in order to realize parallel computing and processing of large amounts of data, a multi-threaded parallel execution solution is generally adopted. In the prior art, in order to improve the problems of excessive overhead in the thread switching process and insufficient parallelism of thread parallel execution, a multi-coroutine technical solution is adopted to improve the efficiency of parallel processing of large amounts of data.

[0003] The inventors have found that in the existing technology for processing data based on multiple coroutines, due to the complex management logic of multiple coroutines, there is still a problem of low efficiency in parallel processing of data. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a network data processing method and device, a server and a storage medium to improve the problem of low data processing efficiency existing in the existing data processing technology.

[0005] To achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0006] A network data processing method, comprising:

[0007] Detecting whether a target data packet exists in a preconfigured task queue through a data receiving coroutine, wherein the data receiving coroutine and the task queue are configured based on a target network thread, and the target network thread is configured with a network coroutine pool including a plurality of network coroutines;

[0008] If the target data packet exists in the task queue, a corresponding network coroutine is determined from a plurality of network coroutines included in the network coroutine pool through the data receiving coroutine, and the network coroutine is placed in a pre-configured network linked list;

[0009] If the target data packet does not exist in the task queue, after switching the data receiving coroutine to the main coroutine of the target network thread, the main coroutine is switched to each network coroutine in the network linked list in turn to process the corresponding target data packet through the network coroutine, and after each network coroutine in the network linked list is switched, the main coroutine is switched to the data receiving coroutine to check the task queue again.

[0010] In a preferred embodiment of the present application, in the above-mentioned network data processing method, the step of switching the main coroutine to each network coroutine in the network linked list in sequence to process the corresponding target data packet through the network coroutine includes:

[0011] Switching the main coroutine to each network coroutine in the network linked list in turn, and determining the data type of the corresponding target data packet through the network coroutine;

[0012] If the target data packet is of real-time computing type, the target data packet is computed and processed through the corresponding network coroutine;

[0013] If the target data packet is of a non-real-time computing type, the target data packet is placed into a business queue pre-configured for the target business thread through a corresponding network coroutine for computing and processing.

[0014] In a preferred embodiment of the present application, in the above-mentioned network data processing method, the step of placing the target data packet into a service queue pre-configured for the target service thread for calculation and processing through the corresponding network coroutine includes:

[0015] The target data packet of the non-real-time computing type is placed into the service queue pre-configured for the target service thread through the corresponding network coroutine, and the target service thread is used to detect whether the service queue contains the target data packet;

[0016] If the target data packet exists in the service queue, the target service thread determines a corresponding service coroutine in a pre-configured service coroutine pool, and puts the service coroutine into a pre-configured service linked list, wherein the service coroutine pool includes multiple service coroutines;

[0017] The target service thread is switched to each service coroutine in the service chain list in turn, so as to process the corresponding target data packet through the service coroutine.

[0018] In a preferred embodiment of the present application, in the above-mentioned network data processing method, the step of placing the target data packet into a service queue pre-configured for the target service thread for calculation and processing through the corresponding network coroutine also includes:

[0019] After the corresponding target data packet is placed into the service queue through the network coroutine, the network coroutine is put into sleep mode;

[0020] After the corresponding target data packet is processed by the business coroutine, the processing result is placed in the task queue, and when the processing result is detected by the data receiving coroutine, the corresponding network coroutine is awakened to process the processing result.

[0021] In a preferred embodiment of the present application, in the above network data processing method, the method further includes:

[0022] If the target data packet does not exist in the task queue, the data receiving coroutine is switched to the main coroutine after a preset time.

[0023] In a preferred embodiment of the present application, in the above network data processing method, the method further includes:

[0024] Putting the pre-created request data packet into the communication queue pre-configured for the target communication thread through the network coroutine;

[0025] The target communication thread is used to detect whether there is a request data packet in the communication queue, and when the request data packet exists, the request data packet is sent to other connected servers.

[0026] In a preferred embodiment of the present application, the above network data processing method further includes:

[0027] After the request data packet is placed into the communication queue through the network coroutine, the network coroutine is put into sleep mode;

[0028] After receiving the response data packet sent by the other server based on the request data packet, the corresponding network coroutine is awakened to process the response data packet.

[0029] The present application also provides a network data processing device, including:

[0030] A data packet detection module, configured to detect whether a target data packet exists in a preconfigured task queue through a data receiving coroutine, wherein the data receiving coroutine and the task queue are configured based on a target network thread, and the target network thread is configured with a network coroutine pool including a plurality of network coroutines;

[0031] A network coroutine processing module, configured to, when the target data packet exists in the task queue, determine a corresponding network coroutine from a plurality of network coroutines included in the network coroutine pool through the data receiving coroutine, and put the network coroutine into a pre-configured network linked list;

[0032] A network coroutine switching module is used to, when the target data packet does not exist in the task queue, after switching the data receiving coroutine to the main coroutine of the target network thread, switch the main coroutine to each network coroutine in the network linked list in turn, so as to process the corresponding target data packet through the network coroutine, and after each network coroutine in the network linked list is switched, switch the main coroutine to the data receiving coroutine to re-check the task queue.

[0033] On the basis of the above, the embodiment of the present application further provides a server, including:

[0034] Memory for storing computer programs;

[0035] The processor connected to the memory is used to execute the computer program to implement the above-mentioned network data processing method.

[0036] On the basis of the above, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the above-mentioned network data processing method is implemented.

[0037] The network data processing method and device, server and storage medium provided by the present application configure the main coroutine, data receiving coroutine and multiple network coroutines for the target network thread, so that the target data packet in the task queue corresponding to the target network thread can be detected, distributed and processed by different coroutines. In this way, by configuring different working logics for different coroutines, the coroutines will not interfere with each other, thereby ensuring that different coroutines process different stages of data processing separately, thereby improving the efficiency of parallel execution of coroutines, and then improving the problem of low data processing efficiency in existing data processing technologies, especially in applications of massive data processing, it has a good application effect.

[0038] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of application interaction of a server provided in an embodiment of the present application.

[0040] Figure 2 A flowchart of the steps included in the network data processing method provided in the embodiment of the present application.

[0041] Figure 3 for Figure 2 Schematic diagram of the flow chart of each sub-step included in step S130.

[0042] Figure 4 for Figure 3 Schematic diagram of the flow chart of the sub-steps included in step S135.

[0043] Figure 5 for Figure 3 Schematic diagram of the flow chart of other sub-steps included in step S135.

[0044] Figure 6 A flowchart of other steps included in the network data processing method provided in the embodiment of the present application.

[0045] Figure 7 A flowchart of other steps included in the network data processing method provided in the embodiment of the present application.

[0046] Figure 8 A block diagram of the functional modules included in the network data processing device provided in an embodiment of the present application.

[0047] Icons: 10 - server; 12 - memory; 14 - processor; 100 - network data processing device; 110 - data packet detection module; 120 - network coroutine processing module; 130 - network coroutine switching module. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0049] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0050] like Figure 1 As shown, an embodiment of the present application provides a server 10. The server 10 may include a memory 12, a processor 14 and a network data processing device 100.

[0051] In detail, the memory 12 and the processor 14 are electrically connected directly or indirectly to realize data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The network data processing device 100 may include at least one software function module that can be stored in the memory in the form of software or firmware. The processor 14 is used to execute an executable computer program stored in the memory 12, for example, the software function module and computer program included in the network data processing device 100, so as to implement the network data processing method provided in the embodiment of the present application.

[0052] Optionally, the memory 12 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0053] Furthermore, the processor 14 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0054] Understandably, Figure 1 The structure shown is for illustration only. The server 10 may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for exchanging information with other devices (such as other servers or terminal devices, etc.).

[0055] Combination Figure 2 The embodiment of the present application also provides a network data processing method, which can be applied to the above-mentioned server 10. The method steps defined in the process related to the network data processing method can be implemented by the server 10. Figure 2The specific process shown is explained in detail.

[0056] Step S110, detecting whether there is a target data packet in a pre-configured task queue through a data receiving coroutine.

[0057] In this embodiment, when the server receives a target data packet sent by other devices, the target data packet can be stored in a pre-configured task queue. In this way, the task queue can be detected by the data receiving coroutine (running the data receiving coroutine) to determine whether the target data packet has been received.

[0058] The data receiving coroutine and the task queue are configured based on the target network thread, that is, the data packets that the target network thread needs to process can be stored in the task queue, and the target network thread is configured with a network coroutine pool including multiple network coroutines.

[0059] Furthermore, when it is detected that the target data packet exists in the task queue, step S120 may be executed; when it is detected that the target data packet does not exist in the task queue, step S130 may be executed.

[0060] Step S120, determining a corresponding network coroutine from a plurality of network coroutines included in the network coroutine pool through the data receiving coroutine, and putting the network coroutine into a pre-configured network linked list.

[0061] In this embodiment, when the target data packet is detected through step S110, the corresponding network coroutine can be determined in the network coroutine pool through the data receiving coroutine (i.e., running the data receiving coroutine), and the network coroutine can be placed in a pre-configured network linked list (i.e., using the network coroutine as a data processing coroutine to process the target data packet), so that the network coroutine can be used to process the target data packet during runtime.

[0062] Step S130, after switching the data receiving coroutine to the main coroutine of the target network thread, switching the main coroutine to each network coroutine in the network linked list in turn, so as to process the corresponding target data packet through the network coroutine, and after each network coroutine in the network linked list is switched, switching the main coroutine to the data receiving coroutine to detect the task queue again.

[0063] In this embodiment, when the target data packet is not detected through step S110, the data receiving coroutine can be first switched to the main coroutine of the target network thread, and then the main coroutine is switched to (i.e., traversed) each network coroutine in the network linked list in turn, so that each running network coroutine processes the corresponding target data packet respectively, and after each network coroutine in the network linked list is switched, the main coroutine is switched to the data receiving coroutine to detect the task queue again, thereby completing a cycle.

[0064] Based on the above method, by configuring the main coroutine, data receiving coroutine and multiple network coroutines for the target network thread, the target data packet in the task queue corresponding to the target network thread can be detected, distributed and processed by different coroutines. In this way, by configuring different working logics for different coroutines, the coroutines will not interfere with each other, thereby ensuring that different coroutines process different stages of data processing separately, thereby improving the efficiency of parallel execution of coroutines, and further improving the problem of low data processing efficiency in existing data processing technologies.

[0065] It should be noted that for step S110 , the specific method of detecting the task queue is not limited and can be selected according to actual application requirements.

[0066] For example, in an alternative example, the data receiving coroutine may detect the task queue at intervals of a preset duration to determine whether the target data packet is received.

[0067] The specific value of the preset duration is not limited, for example, it can be configured according to the average QPS (Queries Per Second) of the server, such as 50mS.

[0068] For another alternative example, the target data packet can be detected based on other threads or coroutines, and when the target data packet is detected, the data receiving coroutine is notified, so that the data receiving coroutine can determine whether the target data packet is received based on whether the notification is received.

[0069] Wherein, the specific object of the above-mentioned other threads or coroutines is not limited, for example, it can be a network listening thread (epoll). The network listening thread and the target network thread can be located in the same network thread pool, so that when the network listening thread monitors the corresponding port, if it is detected that a data packet has arrived, a complete data packet can be read through a socket (socket), and the data packet is stored in the task queue corresponding to the port. Then, the network listening thread can notify the target network thread, so that the target network thread can notify the data receiving coroutine, so that when the data receiving coroutine receives the notification of the target network thread, it can determine that there is a target data packet in the task queue.

[0070] It can be understood that the target data packet in the task queue is detected by the data receiving coroutine, and the entrance of the data receiving coroutine can be pointed to the entrance of a pre-created data receiving function, so that when the data receiving coroutine is running, it can detect whether there is a target data packet in the task queue based on the data receiving function.

[0071] It should be noted that for step S120, the specific method of determining the corresponding network coroutine in the network coroutine pool is not limited and can be selected according to actual application requirements.

[0072] For example, in an alternative example, if all network coroutines in the network coroutine pool are of the same type, then an idle network coroutine can be arbitrarily determined in the network coroutine pool, and the network coroutine can be used as the data processing coroutine of the target data packet to process the target data packet.

[0073] For another example, if different network coroutines in the network coroutine pool are of different types and are used to process different data packets, a network coroutine of a corresponding type and in an idle state can be determined in the network coroutine pool based on the data packet, and the network coroutine is used as the data processing coroutine of the target data packet to process the target data packet.

[0074] Furthermore, in step S120, the specific method of placing the determined network coroutine into the network linked list is not limited and can be selected according to actual application requirements.

[0075] For example, in an alternative example, if it is only necessary to identify whether the network coroutine is in an idle state, the network linked list can be an available linked list. In this way, as long as any network coroutine is determined as a data processing coroutine for a target data packet, the network coroutine can be put into the available linked list to indicate that the network coroutine target is not in an idle state and needs to wait for execution to process the target data packet.

[0076] For another example, in another alternative example, considering that the processing mechanisms of different data packets may be different, the corresponding network coroutines may be in different processing states. In this way, in order to improve the management efficiency of network coroutines, the network linked list may include a free linked list, an available linked list, an unavailable linked list, a timeout check linked list, and an active linked list.

[0077] Among them, in the initial default state, all network coroutines in the network coroutine pool can be put into the idle linked list, that is, all network coroutines are in an idle state at the beginning. In this way, when the target data packet is received, it can be determined whether the target data packet is set with a waiting time.

[0078] On the one hand, when the target data packet is not set with a waiting time, the determined network coroutine can be placed in the above-mentioned available linked list. Based on this, for the network coroutine placed in the available linked list, if the network coroutine needs to wait for the processing results of other coroutines after processing the data packet, at this time, the network coroutine can be put into sleep mode (specific examples can be referred to the description below), and during the sleep period, the network coroutine can be placed in the above-mentioned unavailable linked list.

[0079] On the other hand, when the target data packet is provided with a waiting time, the determined network coroutine can be placed in the above-mentioned timeout check linked list. Based on this, for the network coroutine placed in the timeout check linked list, the network coroutine can be placed in the above-mentioned active linked list after the waiting time is met.

[0080] Among them, for the network coroutines placed in the available linked list and the active linked list, when the main coroutine switches to (traverses) the available linked list and the active linked list in turn, the network coroutine can process the corresponding target data packet, and after processing, the network coroutine is placed in the above-mentioned free linked list. In addition, for the network coroutines that need to wait for the processing results of other coroutines after processing, after the network coroutine is placed in the unavailable linked list, if the processing result is obtained, the network coroutine can be awakened, and the awakened network linked list is placed in the above-mentioned free linked list.

[0081] It can be understood that, in the above example, putting the network coroutine into different linked lists may mean pointing the coroutine pointer of the network coroutine to different linked lists.

[0082] It should be noted that for step S130 , the specific method of processing the corresponding target data packet through the network coroutine is not limited and can be selected according to actual application requirements.

[0083] For example, in an alternative example, the corresponding target data packet can be directly calculated and processed based on the network coroutine. For another example, in another alternative example, the corresponding target data packet can be directly sent to other coroutines for processing through the network coroutine.

[0084] For another example, in order to further improve the efficiency of processing the data packet, in combination with Figure 3 Step S130 may include step S131, step S133 and step S135, the specific contents of which are described as follows.

[0085] Step S131, switching the main coroutine to each network coroutine in the network linked list in turn, and determining the data type of the corresponding target data packet through the network coroutine.

[0086] In this embodiment, after the data receiving coroutine is switched to the main coroutine of the target network thread, the main coroutine may be switched to (traversed) each network coroutine in the network linked list (such as the available linked list and the active linked list mentioned above) in turn. Then, by running each network coroutine switched to, the data type of the corresponding target data packet may be determined, such as determining whether the target data packet belongs to a real-time computing type or a non-real-time computing type.

[0087] When it is determined that the data type of the corresponding target data packet is a real-time calculation type, step S133 may be executed; when it is determined that the data type of the corresponding target data packet is a non-real-time calculation type, step S135 may be executed.

[0088] Step S133, performing calculation processing on the target data packet through the corresponding network coroutine.

[0089] In this embodiment, when it is determined in step S131 that the data type of the target data packet is a real-time calculation type, in order to ensure that the target data packet can be calculated and processed in a timely manner, the target data packet can be directly calculated and processed through the corresponding network coroutine.

[0090] Step S135, putting the target data packet into a service queue pre-configured for the target service thread through the corresponding network coroutine for calculation and processing.

[0091] In this embodiment, when the data type of the target data packet is determined to be a non-real-time calculation type through step S131, for the overall efficiency of data processing, such as to ensure that other data packets can also be calculated and processed in a timely and effective manner, the target data packet can be placed in a business queue pre-configured for the target business thread for calculation and processing.

[0092] Among them, after the corresponding target data packet is placed in the service queue through the network coroutine, if there is no need to wait for the processing result, the network coroutine can be placed in the above-mentioned free linked list; if it is necessary to wait for the processing result, the network coroutine can be placed in the above-mentioned unavailable linked list.

[0093] Furthermore, the target service thread may be a service thread among a plurality of service threads included in a pre-configured service thread pool.

[0094] It can be understood that, in the above examples, the specific contents of the real-time calculation type and the non-real-time calculation type can be pre-configured according to actual application requirements.

[0095] For example, in an alternative example, the non-real-time computing type may include a parallel processing type (such as the need to update multiple databases in parallel), a time-consuming type, and a type that requires sleep processing, and the real-time computing type may refer to other types that are short in time, simple in calculation, and have a small amount of data.

[0096] Optionally, the specific manner of executing step S135 to put the target data packet into the service queue for calculation processing is not limited and can be selected according to actual application requirements.

[0097] For example, in an alternative example, the target data packet may be directly calculated and processed by the target service thread.

[0098] For example, in another alternative example, in order to avoid problems such as excessive resource consumption, the target data packet can also be calculated and processed by a service coroutine configured for the target service thread. Figure 4 , step S135 may include step S135a, step S135b and step S135c, the specific contents of which are described as follows.

[0099] Step S135a, putting the target data packet of non-real-time computing type into a business queue pre-configured for the target business thread through the corresponding network coroutine, and detecting whether the business queue contains the target data packet through the target business thread.

[0100] In this embodiment, when it is determined based on step S131 that the data type of the target data packet belongs to the non-real-time computing type, the target data packet can be placed in a service queue pre-configured for the target service thread through the corresponding network coroutine. At the same time, the service queue can be detected by the target service thread to determine whether the target data packet exists in the service queue.

[0101] Among them, when it is determined through the target business thread that the target data packet exists in the business queue, step S135b can be executed; when it is determined through the target business thread that the target data packet does not exist in the business queue, after waiting for a certain period of time, it can be checked again whether there is a target data packet in the business queue.

[0102] Step S135b, determining the corresponding business coroutine in the pre-configured business coroutine pool through the target business thread, and putting the business coroutine into the pre-configured business linked list.

[0103] In this embodiment, when it is determined through step S135a that there is a target data packet in the business queue, the corresponding business coroutine can be determined in the business coroutine pool through the target business thread (i.e., running the target business thread), and the business coroutine can be placed in a pre-configured business linked list, that is, the business coroutine is used as the business processing coroutine of the target data.

[0104] The business coroutine pool is configured based on the target business thread, and the business coroutine pool includes multiple business coroutines, so that multiple data packets can be processed separately.

[0105] Step S135c, switching the target service thread to each service coroutine in the service linked list in turn, so as to process the corresponding target data packet through the service coroutine.

[0106] In this embodiment, after the determined business coroutine is placed into the business linked list based on step S135b, the target business thread can be switched to (traversed) each business coroutine in the business linked list in turn, so that each business coroutine in the business linked list can be run to process the corresponding target data packet.

[0107] Among them, the above-mentioned business linked list can be used to characterize the current state of the business coroutine, such as the above-mentioned network linked list, and can also include an idle linked list, an available linked list, an unavailable linked list, a timeout check linked list and an active linked list, and the represented state can be referred to the previous description, which will not be repeated here.

[0108] Furthermore, in order to avoid the problem of wasting resources, after executing step S135a, in an alternative example, Figure 5 , step S135 may also include step S135d and step S135e, the specific contents of which are described as follows.

[0109] Step S135d, put the network coroutine into sleep mode.

[0110] In this embodiment, after executing step S135a to put the corresponding target data packet into the service queue through the network coroutine, the network coroutine may be put to sleep to avoid consumption of memory resources by the network coroutine.

[0111] Step S135e, putting the processing result into the task queue, and when the processing result is detected by the data receiving coroutine, waking up the corresponding network coroutine to process the processing result.

[0112] In this embodiment, after executing step S135c to process the corresponding target data packet through the business coroutine, the processing result can be obtained and placed in the task queue, so that the data receiving coroutine can obtain the processing result when detecting the task queue.

[0113] Furthermore, when the processing result is obtained through the data receiving coroutine, the corresponding (sleeping) network coroutine can be awakened so that the network coroutine can process the processing result after awakening.

[0114] For example, in a specific application example, the target data packet corresponding to the processing result is to update the database. If the processing result is that the database update is completed, the network coroutine can generate update success information and send it to the terminal device corresponding to the target data packet; if the processing result is that the database update fails, the network coroutine can generate update failure or re-update information and send it to the terminal device corresponding to the target data packet, so that the terminal device can resend the request for update processing.

[0115] Furthermore, considering that the premise of executing step S130 is to first switch the data receiving coroutine to the main coroutine of the target network thread, the network data processing method may further include the step of switching the data receiving coroutine to the main coroutine of the target network thread.

[0116] This step may be configured differently based on different requirements.

[0117] For example, in an alternative example, when it is detected that the target queue does not exist in the task queue, the data receiving coroutine can be immediately switched to the main coroutine.

[0118] For example, in another alternative example, in order to avoid the problem of resource waste caused by frequent switching, when it is detected that the target queue does not exist in the task queue, the data receiving coroutine can be switched to the main coroutine after a preset time (which can be configured according to actual application requirements, such as 2, 4, 5, 7, etc. S).

[0119] Furthermore, based on some application requirements, the server needs to communicate with other servers. In order to enable the server to request other servers to perform corresponding services based on the generated request data packet, in this embodiment, combined with Figure 6 The network data processing method may further include step S140 and step S150, the specific contents of which are described as follows.

[0120] Step S140, placing the pre-created request data packet into a communication queue pre-configured for the target communication thread through the network coroutine.

[0121] In this embodiment, when it is necessary to request other servers to provide services, the pre-created request data packet can be placed into the pre-configured communication queue through the network coroutine (ie, running the network coroutine).

[0122] The communication queue is formed based on the configuration of a target communication thread, and the target communication thread may be a communication thread in a pre-created communication thread pool.

[0123] Step S150, detecting whether there is a request data packet in the communication queue through the target communication thread, and sending the request data packet to other connected servers if there is the request data packet.

[0124] In this embodiment, after the request data packet is placed into the communication queue based on step S140, since the target communication thread can detect the communication queue, when it is detected that there is a request data packet in the communication queue, the request data packet can be sent to other servers, so that the other servers process the request data packet, that is, let the other servers provide corresponding services.

[0125] It should be noted that for step S140, the method for creating the request data packet is not limited and can be selected according to actual application requirements.

[0126] For example, in an alternative example, if the request data packet to be created is relatively simple, it can be created directly based on the network coroutine. For another example, in another alternative example, if the request data packet to be created is relatively complex (such as a large amount of data or complex logic), it can be created through a business coroutine.

[0127] What needs to be explained about step S150 is that, when the target communication thread detects that there is a request data packet in the communication queue, it can first request other servers to establish a connection, and put the request data packet into a pre-established timeout check queue, and then, after establishing a connection with other servers, in an alternative example, the request data packet can be directly sent to the other servers; in another alternative example, the request data packet can also be copied from the timeout check queue and the copied request data packet can be sent to other servers.

[0128] Furthermore, after executing step S140 to send the request data packet to other servers, under some application requirements, it is not necessary to wait for other servers to return service results, while under other application requirements, it is possible to wait for other servers to return service results.

[0129] In the case where other servers are required to return service results, the network coroutine corresponding to the request data packet can be processed differently based on different requirements.

[0130] For example, in an alternative example, in order to avoid the problem of resource waste, the network coroutine corresponding to the request data packet can be put into sleep mode. Figure 7 The network data processing method may further include step S160 and step S170, the specific contents of which are described as follows.

[0131] Step S160, after the request data packet is placed into the communication queue through the network coroutine, the network coroutine is put into sleep mode.

[0132] In this embodiment, after the request data packet is placed into the communication queue through the network coroutine based on step S140, the network coroutine may be put to sleep.

[0133] Step S170, after receiving the response data packet sent by the other server based on the request data packet, wake up the corresponding network coroutine to process the response data packet.

[0134] In this embodiment, after the network coroutine is put to sleep based on step S160 and the request data packet is sent to other servers based on step S150, it is possible to wait for other servers to return service results.

[0135] Among them, other servers can return service results by sending a response data packet to the server. In this way, when the target communication thread detects the receipt of the response data packet, the response data packet can be placed in the task queue, so that when the data receiving coroutine detects the response data packet in the task queue, the corresponding network coroutine can be awakened to process the response data packet.

[0136] It should be noted that, in the above example, putting a coroutine (such as a network coroutine) to sleep means suspending the coroutine for a period of time (the specific duration is configured according to actual application requirements).

[0137] Combination Figure 8 The embodiment of the present application further provides a network data processing device 100, which can be applied to the above-mentioned server 10. The network data processing device 100 may include a data packet detection module 110, a network coroutine processing module 120 and a network coroutine switching module 130.

[0138] The data packet detection module 110 is used to detect whether there is a target data packet in a pre-configured task queue through a data receiving coroutine, wherein the data receiving coroutine and the task queue are configured based on a target network thread, and the target network thread is configured with a network coroutine pool including multiple network coroutines. In this embodiment, the data packet detection module 110 can be used to perform Figure 2 As shown in step S110, the relevant contents of the data packet detection module 110 can refer to the above description of step S110.

[0139] The network coroutine processing module 120 is used to determine the corresponding network coroutine from the multiple network coroutines included in the network coroutine pool through the data receiving coroutine when the target data packet exists in the task queue, and put the network coroutine into a pre-configured network linked list. In this embodiment, the network coroutine processing module 120 can be used to execute Figure 2 As shown in step S120, the relevant content of the network coroutine processing module 120 can refer to the above description of step S120.

[0140] The network coroutine switching module 130 is used to switch the data receiving coroutine to the main coroutine of the target network thread, and then switch the main coroutine to each network coroutine in the network linked list in sequence when the target data packet does not exist in the task queue, so as to process the corresponding target data packet through the network coroutine, and after each network coroutine in the network linked list is switched, switch the main coroutine to the data receiving coroutine to detect the task queue again. In this embodiment, the network coroutine switching module 130 can be used to perform Figure 2As shown in step S130, the relevant content of the network coroutine switching module 130 can refer to the above description of step S130.

[0141] In an embodiment of the present application, corresponding to the above-mentioned network data processing method, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the above-mentioned network data processing method is executed.

[0142] Among them, the steps executed when the aforementioned computer program is running will not be described one by one here, and reference can be made to the explanation of the aforementioned network data processing method in the previous text.

[0143] It should be noted that in the description of the above examples, "multiple" refers to two or more.

[0144] In summary, the network data processing method and device, server and storage medium provided by the present application configure the main coroutine, data receiving coroutine and multiple network coroutines for the target network thread, so that the target data packet in the task queue corresponding to the target network thread can be detected, distributed and processed by different coroutines. In this way, by configuring different working logics for different coroutines, the coroutines will not interfere with each other, thereby ensuring that different coroutines process different stages of data processing separately, thereby improving the efficiency of parallel execution of coroutines, and then improving the problem of low data processing efficiency in existing data processing technologies, especially in applications of massive data processing, it has a good application effect.

[0145] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0146] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0147] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0148] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A network data processing method, characterized in that: include: Detecting whether a target data packet exists in a preconfigured task queue through a data receiving coroutine, wherein the data receiving coroutine and the task queue are configured based on a target network thread, and the target network thread is configured with a network coroutine pool including a plurality of network coroutines; If the target data packet exists in the task queue, a corresponding network coroutine is determined from a plurality of network coroutines included in the network coroutine pool through the data receiving coroutine, and the network coroutine is placed in a pre-configured network linked list; If the target data packet does not exist in the task queue, after switching the data receiving coroutine to the main coroutine of the target network thread, the main coroutine is switched to each network coroutine in the network linked list in sequence, so as to process the corresponding target data packet by running each network coroutine switched to, and after each network coroutine in the network linked list is switched, the main coroutine is switched to the data receiving coroutine to detect the task queue again; The step of sequentially switching the main coroutine to each network coroutine in the network linked list, so as to process the corresponding target data packet by running each network coroutine switched to, comprises: Switching the main coroutine to each network coroutine in the network linked list in turn, and determining the data type of the corresponding target data packet by running each network coroutine switched to; If the target data packet is of real-time computing type, the target data packet is computed and processed through the corresponding network coroutine; If the target data packet is of a non-real-time computing type, the target data packet is placed into a business queue pre-configured for the target business thread through a corresponding network coroutine for computing and processing.

2. The network data processing method according to claim 1, characterized in that: The step of placing the target data packet into a service queue pre-configured for the target service thread for calculation and processing through the corresponding network coroutine includes: Put the target data packet of non-real-time computing type into the business queue pre-configured for the target business thread through the corresponding network coroutine, and detect whether the business queue contains the target data packet through the target business thread; If the target data packet exists in the service queue, the target service thread determines a corresponding service coroutine in a pre-configured service coroutine pool, and puts the service coroutine into a pre-configured service linked list, wherein the service coroutine pool includes multiple service coroutines; The target service thread is switched to each service coroutine in the service chain list in turn, so as to process the corresponding target data packet through the service coroutine.

3. The network data processing method according to claim 2, characterized in that: The step of placing the target data packet into a service queue pre-configured for the target service thread for calculation and processing through the corresponding network coroutine also includes: After the corresponding target data packet is placed into the service queue through the network coroutine, the network coroutine is put into sleep mode; After the corresponding target data packet is processed by the business coroutine, the processing result is placed in the task queue, and when the processing result is detected by the data receiving coroutine, the corresponding network coroutine is awakened to process the processing result.

4. The network data processing method according to claim 1, characterized in that: The method further comprises: If the target data packet does not exist in the task queue, the data receiving coroutine is switched to the main coroutine after a preset time.

5. The network data processing method according to any one of claims 1 to 4, characterized in that: The method further comprises: Putting the pre-created request data packet into the communication queue pre-configured for the target communication thread through the network coroutine; The target communication thread is used to detect whether there is a request data packet in the communication queue, and when the request data packet exists, the request data packet is sent to other connected servers.

6. The network data processing method according to claim 5, characterized in that: Also includes: After the request data packet is placed into the communication queue through the network coroutine, the network coroutine is put into sleep mode; After receiving the response data packet sent by the other server based on the request data packet, the corresponding network coroutine is awakened to process the response data packet.

7. A network data processing device, characterized in that: include: A data packet detection module, configured to detect whether a target data packet exists in a preconfigured task queue through a data receiving coroutine, wherein the data receiving coroutine and the task queue are configured based on a target network thread, and the target network thread is configured with a network coroutine pool including a plurality of network coroutines; A network coroutine processing module, configured to, when the target data packet exists in the task queue, determine a corresponding network coroutine from a plurality of network coroutines included in the network coroutine pool through the data receiving coroutine, and put the network coroutine into a pre-configured network linked list; A network coroutine switching module is used for, when the target data packet does not exist in the task queue, after switching the data receiving coroutine to the main coroutine of the target network thread, switching the main coroutine to each network coroutine in the network linked list in sequence, so as to process the corresponding target data packet by running each network coroutine switched to, and after each network coroutine in the network linked list is switched, switching the main coroutine to the data receiving coroutine, so as to detect the task queue again; The network coroutine switching module is specifically used to switch the main coroutine to each network coroutine in the network linked list in turn, and determine the data type of the corresponding target data packet by running each network coroutine switched to; if the target data packet is a real-time calculation type, the target data packet is calculated and processed by the corresponding network coroutine; if the target data packet is a non-real-time calculation type, the target data packet is placed in a business queue pre-configured for the target business thread through the corresponding network coroutine for calculation and processing.

8. A server, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute the computer program to implement the network data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the network data processing method described in any one of claims 1 to 6 is implemented.

Images