Network data processing method, device and storage medium

Abstract

The embodiments of the present application provide a method, device and storage medium for processing network data. In the embodiments of the present application, network data is sent to the first preset areas corresponding to each other through at least one first thread; at least one first thread corresponds to one second thread; when the second thread is in an idle state, and the second thread detects that there is network data in any first preset area, the network data is obtained from the first preset area, and the network data is processed, and the processing result is sent to the corresponding second preset area; since at least one first thread corresponds to one second thread, and the second thread will perform operations only in an idle state, when there is continuously data to be processed, the second thread will continue to perform data processing operations, so that the time for the second thread to independently execute operations is greatly improved, without being restricted by whether the first thread is executing operations, thereby improving data processing performance and completing the data processing function at the same time.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and storage medium for processing network data. Background Art

[0002] With the development of information technology, in traditional business scenarios, the requirements for resource access and response time are not high, and databases can usually meet business needs. However, with the sharp increase in business access, users' requirements for product experience have increased, which will inevitably put great pressure on resources. At this time, the database cannot meet business needs. Especially in the scenario of Internet business, the access volume is often dozens or even hundreds of times the original access volume. Therefore, simply optimizing the database layer can no longer meet the needs, but putting data in memory can provide faster reading efficiency. Summary of the invention

[0003] Multiple aspects of the present application provide a network data processing method, device, and storage medium to conveniently and quickly improve the performance of a storage system.

[0004] An embodiment of the present application provides a method for processing network data, comprising: when network data is detected, sending the network data to respective corresponding first preset areas through at least one first thread; wherein the at least one first thread corresponds to a second thread; when the second thread is in an idle state, and the presence of the network data in any of the first preset areas is detected through the second thread, the network data in the first preset area is processed through the second thread, and the processing result is sent to the corresponding second preset area; after completing sending the processing result, the second thread is in an idle state.

[0005] An embodiment of the present application also provides a method for processing network data, including: creating at least one first thread and a second thread corresponding to the at least one first thread, and configuring corresponding functions for the first thread and the second thread; wherein the first thread is used to receive network data from the network transmission channel it is responsible for, and send the network data to the corresponding first preset area, and the second thread is used to obtain the network data from the first preset area when it is in an idle state, and process the network data, and send the processing result to the corresponding second preset area, and the first thread is also used to send the processing result in the corresponding second preset area to the corresponding network transmission channel.

[0006] An embodiment of the present application also provides a computing device, including a memory, a processor, and a communication component; the memory is used to store a computer program; the processor is used to execute the computer program, so as to: when network data is detected, send the network data to the respective corresponding first preset areas through at least one first thread; wherein the at least one first thread corresponds to a second thread; when the second thread is in an idle state, and the presence of the network data in any of the first preset areas is detected by the second thread, the network data in the first preset area is processed by the second thread, and the processing result is sent to the corresponding second preset area; after completing the sending of the processing result, the second thread is in an idle state.

[0007] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors implement the steps in the above-mentioned network data processing method.

[0008] An embodiment of the present application also provides a computing device, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to: create at least one first thread and a second thread corresponding to the at least one first thread, and configure corresponding functions for the first thread and the second thread; wherein the first thread is used to receive network data from the network transmission channel it is responsible for, and send the network data to the corresponding first preset area, the second thread is used to obtain the network data from the first preset area when it is in an idle state, and process the network data, and send the processing result to the corresponding second preset area, and the first thread is also used to send the processing result in the corresponding second preset area to the corresponding network transmission channel.

[0009] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors implement the steps in the above-mentioned network data processing method.

[0010] In an embodiment of the present application, network data is sent to the first preset areas corresponding to each other through at least one first thread; at least one first thread corresponds to one second thread; when the second thread is in an idle state, and the second thread detects the presence of network data in any first preset area, the second thread obtains the network data from the first preset area, processes the network data, and sends the processing result to the corresponding second preset area; since at least one first thread corresponds to one second thread, and the second thread will only perform operations in an idle state, when there is continuous data to be processed, the second thread will continue to perform data processing operations, so that the independent execution time of the second thread is greatly improved, without being restricted by whether the first thread is performing operations, thereby improving data processing performance and completing the data processing function at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1A A schematic diagram of the structure of a network data processing system according to an exemplary embodiment of the present application;

[0013] Figure 1B A schematic diagram of the structure of a network data processing system according to an exemplary embodiment of the present application;

[0014] Figure 2 A flowchart of a method for processing network data according to an exemplary embodiment of the present application;

[0015] Figure 3 A schematic diagram of the structure of a network data processing device provided by an exemplary embodiment of the present application;

[0016] Figure 4 A schematic diagram of the structure of a network data processing device provided by another exemplary embodiment of the present application;

[0017] Figure 5 A schematic diagram of the structure of a server provided for another exemplary embodiment of the present application;

[0018] Figure 6 A schematic diagram of the structure of a server provided for yet another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] The Redis storage system is a key-value storage system. It supports storing more value types, including string, list, set, zset (sorted set) and hash. The Redis storage system supports various sorting methods. To ensure efficiency, data is cached in memory. The Redis storage system periodically writes updated data to disk or writes modification operations to additional record files, and implements master-slave synchronization on this basis.

[0021] In traditional multi-threaded solutions, the Redis storage system is usually split into multiple identical threads, that is, each thread still handles the same operation. However, this goes against the original design intention of the Redis storage system because they split the logic of command execution into a parallel mode, destroying the serial logic of command execution and not being fully compatible with the original functions of Redis, such as transactions, Lua scripts, and other functions.

[0022] In an embodiment of the present application, network data is sent to the first preset areas corresponding to each other through at least one first thread; at least one first thread corresponds to one second thread; when the second thread is in an idle state, and the second thread detects the presence of network data in any first preset area, the second thread obtains the network data from the first preset area, processes the network data, and sends the processing result to the corresponding second preset area; since at least one first thread corresponds to one second thread, and the second thread will only perform operations in an idle state, when there is continuous data to be processed, the second thread will continue to perform data processing operations, so that the independent execution time of the second thread is greatly improved, without being restricted by whether the first thread is performing operations, thereby improving data processing performance and completing the data processing function at the same time.

[0023] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0024] Figure 1A The following is a schematic diagram of a performance improvement system provided by an exemplary embodiment of the present application. Figure 1AAs shown, the lifting system 100A may include: a first device 101 and a second device 102 .

[0025] Among them, the first device 101 can be any computing device with a certain computing capability. The basic structure of the first device 101 may include: at least one processor. The number of processors depends on the configuration and type of the first device 101. The first device 101 may also include a memory, which may be volatile, such as RAM, or non-volatile, such as read-only memory (ROM), flash memory, etc., or may include both types at the same time. The memory usually stores one or more application programs, and may also store program data, etc. In addition to the processing unit and the memory, the first terminal 101 may also include some basic configurations, such as a network card chip, an IO bus, a display component, and some peripheral devices, etc. Optionally, some peripheral devices may include, for example, a keyboard, a mouse, an input pen, a printer, etc. Other peripheral devices are well known in the art and will not be described in detail here.

[0026] The second device 102 refers to a device that can provide computing and processing services in a network virtual environment, and generally refers to a server that uses the network to store information. In physical implementation, the second device 102 can be any device that can provide computing services, respond to service requests, and perform processing, such as a conventional server, a cloud server, a cloud host, a virtual center, etc. The second device 102 mainly includes a processor, a hard disk, a memory, a system bus, etc., which is similar to a general computer architecture.

[0027] In the example of the present application, the first device 101 sends the network data to be stored to the second device 102 through the network transmission channel. The second device 102 is provided with a Redis storage system. When the second device 102 detects the network data, it sends the network data to the first preset area corresponding to each other through at least one first thread; at least one first thread corresponds to one second thread; when the second thread is in an idle state, and the existence of network data in any first preset area is detected through the second thread, the network data in the first preset area is processed through the second thread, and the processing result is sent to the corresponding second preset area; after completing the sending of the processing result, the second thread is in an idle state.

[0028] The second device 102 returns the processing result in the second preset area to the first device 101 through the first thread corresponding to any first preset area.

[0029] In some examples, the first device 101 may have multiple devices and may send network data to the second device 102 at the same time.

[0030] In this embodiment, if Figure 1B As shown, the first device 101, such as a terminal, has an online shopping client installed therein, in response to a shopping account registration request from a user, such as an online buyer, to send attribute data of the registered account, such as user ID, age, gender, etc., to the second device 102, such as a server. After receiving the attribute data, the second device 102 allocates the attribute data to the corresponding first thread. The first thread, such as an IO thread, writes data, such as storing the attribute data in a first storage area, such as a first message queue. The second thread, such as a Worker thread, polls the first message queue corresponding to each IO thread when it is idle. When it is found that there is a message to be processed in the first message queue, the second thread 102 sends the attribute data to the corresponding first thread. When, that is, these attribute data, the Worker thread obtains these attribute data from the first message queue. At this time, the second thread is in a busy state, that is, a non-idle state, and processes these attribute data, such as storing them in the storage table corresponding to the Redis storage system, and then sends the processing results, such as successfully stored messages, to the second message queue corresponding to the IO thread. At this time, the second thread returns to the idle state, and each IO thread polls the second message queue it is responsible for, and sends the messages in the polled second message queue, such as successfully stored messages, to the corresponding terminal, that is, the first device 101, to inform the online shopping client installed on the terminal that the storage is successful.

[0031] It should be noted that the number of the first threads is at least one, and the specific value can be set according to business requirements, so that the time for the second thread to execute the operation is improved, and the number of the second threads is one.

[0032] In the above embodiment, the first device 101 can be connected to the second device 102 through a network, and the network connection can be a wireless or wired network connection. If the first device 101 and the second device 102 are connected in communication, the network standard of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, etc.

[0033] The data processing process of the second device 102 is described in detail below in conjunction with the method embodiment.

[0034] Figure 2 The flowchart of the method for processing network data of an exemplary embodiment of the present application is shown in FIG. The method 200 provided in the embodiment of the present application can be executed by a second device, such as a server, and the method 200 includes the following steps:

[0035] 201: When network data is detected, the network data is sent to the first preset areas corresponding to each other through at least one first thread; wherein at least one first thread corresponds to one second thread.

[0036] 202: When the second thread is in an idle state and the second thread detects that there is network data in any first preset area, the second thread processes the network data in the first preset area and sends the processing result to the corresponding second preset area.

[0037] 203: After completing the sending of the processing result, the second thread is in an idle state.

[0038] It should be noted that this embodiment can be applied to the Redis storage system; the Redis storage system is an open source software, an in-memory database, and supports data types such as strings, hash tables, lists, sets, ordered sets, bitmaps, hyperloglogs, etc.; the Redis storage system can also have built-in replication, Lua scripts, LRU (Least Recently Used page built-in algorithm) retrieval, transactions, and different levels of disk persistence functions.

[0039] The following is a detailed description of the above steps:

[0040] 201: When network data is detected, the network data is sent to the first preset areas corresponding to each other through at least one first thread; wherein at least one first thread corresponds to one second thread.

[0041] Network data refers to data stored in the storage system of the server through a network transmission channel. The network transmission channel refers to a channel used to send data or messages, such as a Socket connection channel.

[0042] The first preset area refers to a storage area used to temporarily store messages to be processed or network data to be processed, such as a message queue or a cache queue and the like.

[0043] The first thread is used to receive network data from the responsible network transmission channel and send the network data to the corresponding first preset area, for example, the IO thread. The number of the first threads is at least one or more, and the number can be set according to business needs and meet the time to improve the operation of the second thread.

[0044] In some instances, network data is sent to the respective corresponding first preset areas through at least one first thread, including: when network data sent by any network transmission channel is monitored, the first thread responsible for the network transmission channel obtains the network data; and the network data is sent to the first message queue corresponding to the first thread through the first thread.

[0045] For example, the server allocates multiple first threads, such as multiple IO threads, to each of its service ports, and multiple terminals can establish Socket connections with a service port of the server to generate Socket connection channels. When there are three IO threads responsible for the service port 80 of the server, and when nine terminals make Socket connections with the service port 80 of the server, and nine Socket connection channels are generated, the server can evenly distribute these nine Socket connection channels to the three IO threads of the service port 80, and each IO thread is responsible for three Socket connection channels. When any of the three IO threads monitors the online shopping client in the corresponding terminal to send network data, such as online shopping order data, it may include user ID, order number, price, purchase quantity, shipping address and other data. The IO thread responsible for the terminal receives the online shopping order and sends the online shopping order data to the first message queue corresponding to the IO thread.

[0046] It should be noted that each IO thread may have its own identifier, such as a thread number or a thread name, etc., to distinguish other IO threads. It should be understood that when an IO thread is assigned to a corresponding Socket connection channel, the corresponding or responsible data sending terminal of the IO thread is determined.

[0047] If the IO thread responsible for a network transmission channel does not receive network data, the IO thread can be blocked. At this time, the IO thread can first process the network data sent by other network transmission channels. If all the network transmission channels responsible for the IO thread send network data, the IO thread can process them in sequence according to the time sequence of receiving the data. If the time sequence is exactly the same, it can be randomly selected for processing.

[0048] Each IO thread corresponds to a message queue or cache queue, that is, there are as many message queues as there are IO threads.

[0049] In step 201, the IO thread reads network data, calls the read method, and writes data.

[0050] The message queue can be a lock-free queue. In a lock-free queue, readers and writers each maintain a read and write pointer. In the single-producer single-consumer mode, they can read and write concurrently without affecting each other.

[0051] 202: When the second thread is in an idle state and the second thread detects that there is network data in any first preset area, the second thread processes the network data in the first preset area and sends the processing result to the corresponding second preset area.

[0052] The second thread is used to obtain network data from the first preset area, process the network data, and send the processing result to the corresponding second preset area, for example, a Worker thread. The number of the second thread is one.

[0053] The processing result refers to the result after processing the pending message or pending data, such as successful storage in the Redis storage system data table.

[0054] The second preset area refers to a storage area used to temporarily store processing results, such as a message queue or a cache queue, and the like.

[0055] In some instances, the network data in the first preset area is processed by the second thread, and the processing results of the data are sent to the corresponding second preset area, including: when the second thread in an idle state finds that any first message queue in at least one first message queue has network data, the network data is obtained; the network data is parsed, and the network data is processed according to the parsing results; and the processing results are sent to the second message queue corresponding to the first message queue.

[0056] The method of searching for network data in the message queue may be: searching for network data from the message queue through a polling method or mechanism.

[0057] The polling method or mechanism means that the second thread periodically sends inquiries to the message queue, asking each message queue in sequence whether it needs its service. If so, it will provide service. After the service is completed, it will ask the next message queue, and then repeat the cycle.

[0058] Parsing network data refers to parsing network data according to a preset protocol, such as a Redis protocol, to obtain parsing results, such as network data and a method for processing the network data.

[0059] The second thread in an idle state means that the second thread is not currently executing any task or processing any data, and is in a state of waiting to execute a task or process data.

[0060] For example, according to the above, when the second process of the server is in an idle state, such as a Worker thread, the first message queues of all IO threads in the Redis storage system are polled. When any of the first message queues is polled to have pending messages or data, such as online shopping order data, the Worker thread can obtain the online shopping order data from the first message queue. At this time, the second thread, such as the Worker thread, changes from an idle state to a busy state or a processing task state. The Worker thread parses the online shopping data using the Redis protocol and obtains each data in the online shopping order data, as well as an instruction to store each data in the Redis storage system data table. The Worker thread stores each data in the Redis storage system data table according to the instruction. When the data is successfully stored in the Redis storage system data table, the server's Worker thread sends the processing result, "successfully stored in the Redis storage system data table", to the second message queue corresponding to the above IO thread, that is, the second message queue corresponding to the first message queue. The IO thread refers to the IO thread that obtains the online shopping order data. At this time, the second thread, such as the Worker thread, changes from a busy state to an idle state, and can continue to poll the first message queues of all IO threads in the Redis storage system.

[0061] It should be noted that the Worker thread also completes the processing of messages or data in sequence according to the logical order. In addition, in this embodiment, there is only one Worker thread. Since there is only one Worker thread, the IO thread and the Worker thread can process data and messages according to the logical order. However, each logical order is composed of any one IO thread among multiple IO threads and the one Worker thread. Each IO thread corresponds to a second message queue.

[0062] 203: After completing sending the processing result, the second thread is in an idle state.

[0063] Since the specific implementation of step 203 has been described in detail above, it will not be repeated here.

[0064] In some examples, the method 200 further includes: returning the processing result in the second preset area through a first thread corresponding to any first preset area.

[0065] The first thread may also be used to send the processing result of the data to the corresponding second preset area, and send the processing result in the second preset area to the corresponding network transmission channel.

[0066] In some examples, returning the processing result in the second preset area through the first thread corresponding to any first preset area includes: when the corresponding first thread finds that the processing result exists in the first message queue, returning the processing result through the network transmission channel.

[0067] The method of searching for the processing result in the first message queue may be: searching for the processing result from the message queue through a polling method or mechanism.

[0068] For example, according to the foregoing, multiple IO threads of the server poll their own second message queues respectively. When the above IO thread polls that the processing result "successfully stored in the Redis storage system data table" exists in its own second message queue, the IO thread takes the processing result from the second message queue and returns the processing result "successfully stored in the Redis storage system data table" to the client in the corresponding terminal through the Socket connection channel it is responsible for.

[0069] Wherein, at least one first thread runs independently of each other and does not interfere with each other, and the second thread is in a serial relationship with any first thread. As it has been explained in detail above, it will not be repeated here.

[0070] In some instances, the method 200 also includes: creating at least one first thread and a second thread, and configuring corresponding functions for the first thread and the second thread; assigning a network transmission channel for which it is responsible to each first thread, and when multiple network data from different transmission channels are detected, executing the step of sending the network data to the corresponding first preset area through the corresponding first thread.

[0071] For example, according to the foregoing, the server can create multiple IO threads and a Worker thread according to the Redis storage system program, and configure corresponding functions for each IO thread, such as executing commands or executing instructions or operations, so as to realize that when receiving network data from the responsible network transmission channel, the network data is sent to the corresponding first preset area, and the processing result is sent to the corresponding second preset area, and the processing result in the second preset area is sent to the corresponding network transmission channel. And the Worker thread is configured with corresponding functions, such as executing commands or executing instructions or operations, so as to obtain network data from the first preset area and process the network data. When the server has allocated respective network transmission channels to each IO, step 201 can be started. Since the specific functions of each IO thread and Worker thread have been elaborated in detail in the foregoing, they will not be repeated here.

[0072] The native Redis storage system uses a single-process single-thread form when processing events, that is, the processing process is all in one thread, which is also the performance bottleneck. A large amount of CPU time is consumed in data reading and writing, while the actual command execution logic accounts for only 30%; in this embodiment, the Redis storage system is reconstructed into multiple threads, and network IO threads and Worker threads are split. The IO thread can be expanded arbitrarily, and there is only one worker thread to ensure the serial execution of commands, which ensures the performance improvement and functional compatibility of the Redis storage system at the same time. And the performance can be improved to about 3 times the original.

[0073] Another exemplary embodiment of the present application provides another method for processing network data. The method provided in the embodiment of the present application is performed by a second device, such as a server, and the method includes the following steps:

[0074] A: Create at least one first thread and a second thread corresponding to the at least one first thread, and configure corresponding functions for the first thread and the second thread; wherein the first thread is used to receive network data from the network transmission channel it is responsible for, and send the network data to the corresponding first preset area, and the second thread is used to obtain network data from the first preset area when it is in an idle state, and process the network data, and send the processing result to the corresponding second preset area, and the first thread is also used to send the processing result in the corresponding second preset area to the corresponding network transmission channel.

[0075] It should be noted that since step A has been explained in detail in the previous text, it will not be repeated here.

[0076] Figure 3 A schematic diagram of a structural framework of a network data processing device provided as another exemplary embodiment of the present application. The device 300 can be applied to a second device, such as a server, and includes: a sending module 301 and a processing module 302. The functions of each module are described in detail below:

[0077] The sending module 301 is used for, when network data is detected, at least one first thread to send the network data to a first preset area corresponding to each of them; wherein, at least one first thread corresponds to one second thread.

[0078] The processing module 302 is used for, when the second thread is in an idle state and the second thread detects that there is network data in any first preset area, processing the network data in the first preset area through the second thread and sending the processing result to the corresponding second preset area.

[0079] After completing sending the processing result, the second thread is in an idle state.

[0080] In some examples, the apparatus 300 further includes: a returning module, configured to return the processing result in the second preset area through a first thread corresponding to any first preset area.

[0081] In some instances, the device 300 also includes: a creation module, used to create at least one first thread and a second thread, and configure corresponding functions for the first thread and the second thread; an allocation module, used to allocate the network transmission channel for which it is responsible to each first thread, and when multiple network data from different transmission channels are detected, execute the step of sending the network data to the corresponding first preset area through the corresponding first thread.

[0082] In some instances, the sending module 301 includes: an acquisition unit, which is used to monitor the network data sent by any network transmission channel, and the first thread responsible for the network transmission channel acquires the network data; a first sending unit, which is used to send the network data to the first message queue corresponding to the first thread through the first thread.

[0083] In some instances, the processing module 302 includes: a search unit, which is used to obtain network data when a second thread in an idle state finds that any first message queue in at least one first message queue corresponding to at least one first thread contains network data; a parsing unit, which is used to parse the network data and process the network data according to the parsing result; and a second sending unit, which is used to send the processing result to the second message queue corresponding to the first message queue.

[0084] In some examples, the sending module 301 is used to return the processing result through a network transmission channel when finding that there is a processing result in the second message queue corresponding to the first thread through the corresponding first thread.

[0085] In some examples, at least one first thread runs independently of each other; and the second thread is in a serial relationship with any first thread.

[0086] In some instances, the device 300 is applicable to a Redis storage system.

[0087] In some examples, the first thread is an IO thread.

[0088] In some examples, the second thread is a Worker thread.

[0089] Figure 4 A schematic diagram of a structural framework of another network data processing device provided as another exemplary embodiment of the present application. The device 400 can be applied to a second device, such as a server, and the device 400 includes: a creation module 401, and the functions of each module are described in detail below:

[0090] A creation module 401 is provided to create at least one first thread and a second thread corresponding to the at least one first thread, and configure corresponding functions for the first thread and the second thread; wherein the first thread is used to receive network data from the network transmission channel for which it is responsible, and send the network data to the corresponding first preset area; the second thread is used to obtain network data from the first preset area when it is in an idle state, and process the network data, and send the processing result to the corresponding second preset area; the first thread is also used to send the processing result in the corresponding second preset area to the corresponding network transmission channel.

[0091] The above describes Figure 3 The internal functions and structure of the processing device 300 shown in FIG. 3 are shown in FIG. 3 . In one possible design, Figure 3 The structure of the processing device 300 shown can be implemented as a server, such as Figure 5 As shown, the server 500 may include: a memory 501, a processor 502, and a communication component 503;

[0092] Memory 501, used for storing computer programs;

[0093] Processor 502 is used to execute a computer program, so that: when network data is detected, at least one first thread sends the network data to the first preset area corresponding to each other; wherein, at least one first thread corresponds to one second thread; when the second thread is in an idle state, and the second thread detects that there is network data in any first preset area, the second thread processes the network data in the first preset area and sends the processing result to the corresponding second preset area; after completing sending the processing result, the second thread is in an idle state.

[0094] The communication component 503 is used to return the processing result in the second preset area through the first thread corresponding to any first preset area.

[0095] In some instances, the processor 502 is also used to: create at least one first thread and a second thread, and configure corresponding functions for the first thread and the second thread; assign a network transmission channel for which it is responsible to each first thread, and when multiple network data from different transmission channels are detected, execute the step of sending the network data to the corresponding first preset area through the corresponding first thread.

[0096] In some instances, the processor 502 is specifically used to: when monitoring network data sent by any network transmission channel, the first thread responsible for the network transmission channel obtains the network data; and sends the network data to the first message queue corresponding to the first thread through the first thread.

[0097] In some instances, the processor 502 is specifically used to: when a second thread in an idle state finds that network data exists in any first message queue of at least one first message queue corresponding to at least one first thread, obtain the network data; parse the network data, and process the network data according to the parsing result; and send the processing result to the second message queue corresponding to the first message queue.

[0098] In some examples, when the corresponding first thread finds that a processing result exists in the second message queue corresponding to the first thread, the communication component 503 is specifically used to return the processing result through a network transmission channel.

[0099] In some examples, at least one first thread runs independently of each other; and the second thread is in a serial relationship with any first thread.

[0100] In some examples, the processor 502 is adapted for a Redis storage system.

[0101] In some examples, the first thread is an IO thread.

[0102] In some examples, the second thread is a Worker thread.

[0103] In addition, an embodiment of the present invention provides a computer storage medium, when the computer program is executed by one or more processors, causing the one or more processors to implement Figure 2 The steps of the method for processing network data in the method embodiment.

[0104] The above describes Figure 4 The internal functions and structure of the information processing device 400 shown in FIG. 4 are shown in FIG. 4 . In one possible design, Figure 4 The structure of the processing device 400 shown can be implemented as a server, such as Figure 6 As shown, the server 600 may include: a memory 601, a processor 602, and a communication component 603;

[0105] Memory 601, used for storing computer programs;

[0106] Processor 602 is used to execute a computer program to: create at least one first thread and a second thread corresponding to the at least one first thread, and configure corresponding functions for the first thread and the second thread; wherein the first thread is used to receive network data from the network transmission channel for which it is responsible, and send the network data to the corresponding first preset area, and the second thread is used to obtain network data from the first preset area when it is in an idle state, and process the network data, and send the processing result to the corresponding second preset area, and the first thread is also used to send the processing result in the corresponding second preset area to the corresponding network transmission channel.

[0107] In addition, an embodiment of the present invention provides a computer storage medium. When a computer program is executed by one or more processors, the one or more processors are caused to implement step A of the method for processing network data in the embodiment.

[0108] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 201, 202, 203, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0109] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0110] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0111] The present invention is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable multimedia data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable multimedia data processing device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0112] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable multimedia data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0113] These computer program instructions may also be loaded onto a computer or other programmable multimedia data processing device so that a series of operational steps are executed on the computer or other programmable device to produce computer-implemented processing, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0114] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0115] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0116] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing network data, characterized in that: Applicable to the Redis storage system, the method includes: When network data is detected, the network data is sent to the first message queue corresponding to each other through at least one first thread; wherein, the at least one first thread corresponds to a second thread, the network data refers to the data stored in the Redis storage system through the network transmission channel, the first thread is an IO thread, the second thread is a Worker thread, and at least one first thread runs independently of each other without interfering with each other; the second thread is in a serial relationship with any first thread, and the network transmission channels responsible for the first thread all send network data; When the second thread is in an idle state, when the second thread finds that any of the at least one first message queues corresponding to at least one first thread has network data, the network data is acquired; the network data is parsed, and the network data is processed according to the parsing result; and the processing result is sent to the second message queue corresponding to the first message queue; After completing sending the processing result, the second thread is in an idle state.

2. The method according to claim 1, characterized in that The method further comprises: The processing result in the second message queue is returned through the first thread corresponding to any one of the first message queues.

3. The method according to claim 1, characterized in that The method further comprises: Creating the at least one first thread and the second thread, and configuring corresponding functions for the first thread and the second thread; A network transmission channel that the first thread is responsible for is allocated to each of the first threads. When a plurality of network data from different transmission channels are detected, a step of sending the network data to a corresponding first message queue through a corresponding first thread is executed.

4. The method according to claim 1, characterized in that: The sending of the network data to the first message queues corresponding to the respective ones through at least one first thread includes: When network data sent by any network transmission channel is monitored, the first thread responsible for the network transmission channel obtains the network data; The network data is sent to a first message queue corresponding to the first thread through the first thread.

5. The method according to claim 2, characterized in that: The returning the processing result in the second message queue through the first thread corresponding to any one of the first message queues includes: When it is found through the corresponding first thread that there is a processing result in the second message queue corresponding to the first thread, the processing result is returned through the network transmission channel.

6. A method for processing network data, characterized in that: Applicable to the Redis storage system, the method includes: Creating at least one first thread and a second thread corresponding to the at least one first thread, and configuring corresponding functions for the first thread and the second thread; Among them, the first thread is used to receive network data from the network transmission channel it is responsible for, and send the network data to the corresponding first message queue. The second thread is used to, when in an idle state, find that any first message queue corresponding to at least one first thread has network data, and obtain the network data; parse the network data, and process the network data according to the parsing result; send the processing result to the second message queue corresponding to the first message queue; the first thread is also used to send the processing result in the corresponding second message queue to the corresponding network transmission channel. The network data refers to the data stored in the Redis storage system through the network transmission channel. The first thread is an IO thread, and the second thread is a Worker thread. At least one first thread runs independently of each other and does not interfere with each other. The second thread and any first thread are in a serial relationship, and the network transmission channels responsible for the first thread all send network data.

7. A computing device, characterized in that Includes memory, processor, and communication components; The memory is used to store computer programs; The processor is configured to execute the computer program to: When network data is detected, the network data is sent to the first message queues corresponding to each other through at least one first thread; wherein, the at least one first thread corresponds to a second thread; when the second thread is in an idle state, and when the network data exists in any of the at least one first message queues corresponding to at least one first thread is found through the second thread, the network data is obtained; the network data is parsed, and the network data is processed according to the parsing result; the processing result is sent to the second message queue corresponding to the first message queue; the network data refers to the data stored in the Redis storage system through the network transmission channel, the first thread is an IO thread, the second thread is a Worker thread, and at least one first thread runs independently of each other and does not interfere with each other; the second thread and any first thread are in a serial relationship, and the network transmission channels responsible for the first thread all send network data; After completing sending the processing result, the second thread is in an idle state.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the one or more processors are caused to implement the steps of the method according to any one of claims 1 to 5.

9. A computing device, characterized in that including a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program to: Creating a plurality of first threads and second threads corresponding to the plurality of first threads, and configuring corresponding functions for the first threads and the second threads; Among them, the first thread is used to receive network data from the network transmission channel it is responsible for, and send the network data to the corresponding first message queue; the second thread is used to, when in an idle state, find that any first message queue corresponding to at least one first thread has network data, and obtain the network data; parse the network data, and process the network data according to the parsing result; send the processing result to the second message queue corresponding to the first message queue; the first thread is also used to send the processing result in the corresponding second message queue to the corresponding network transmission channel, the network data refers to the data stored in the Redis storage system through the network transmission channel, the first thread is an IO thread, the second thread is a Worker thread, and at least one first thread runs independently of each other without interfering with each other; the second thread and any first thread are in a serial relationship, and the network transmission channels responsible for the first thread all send network data.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the one or more processors are caused to implement the steps of the method according to claim 6.

Images