Inter-process communication method and apparatus for multi-process and multi-thread

Abstract

This invention discloses a multi-process, multi-threaded inter-process communication method and apparatus, relating to the field of computer network technology, which improves communication efficiency and data consistency, and ensures stable operation in complex network full-traffic threat analysis environments. The method includes: determining messages ordered in the lock-free queue bound to each business thread of the sending process; the sending thread in the sending process determines the target message based on the messages ordered in all lock-free queues, determines the target process corresponding to the target message, and determines whether the target process is ready. If ready, the target message is written to the inter-process shared memory area, read by the target process into its memory space, and the receiving thread of the target process parses the target message to obtain its message type and message subject; the receiving thread performs processing operations corresponding to the message type of the target message based on its message subject, thereby completing the multi-process, multi-threaded inter-process communication.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, and in particular to a method and apparatus for inter-process communication involving multiple processes and threads. Background Technology

[0002] With the rapid development of network technology, full-traffic threat analysis has become an important research direction in the field of network security. In this context, systems often need to process large amounts of data streams simultaneously, involving the collaborative work of multiple processes and threads. Traditional inter-process communication methods, such as pipes and message queues, suffer from inefficiency and difficulty in ensuring data consistency in multi-process, multi-threaded environments, making it difficult to meet current demands for high performance and high reliability.

[0003] The open-source event-driven polling mechanism has a single sending interface, limited support for one-to-one and multiple-to-one questions, and is not convenient to use. In multi-threaded environments, it lacks mechanisms to ensure the ordered sending of messages across different threads. In particular, message reliability is not well guaranteed when the other party's process crashes or is starting up and has not yet registered its service interface. Summary of the Invention

[0004] In view of this, the present invention provides a multi-process, multi-threaded inter-process communication method and apparatus to improve communication efficiency and data consistency, and ensure stable operation in a complex network full-traffic threat analysis environment.

[0005] According to one aspect of the present invention, a method for inter-process communication in a multi-process, multi-threaded manner is provided, the method comprising: Determine the order of messages in the lock-free queue bound to each business thread of the sending process; The sending thread in the sending process determines the target message based on the messages sorted in all the lock-free queues, performs a dequeue operation, stores the target message in the working buffer of the sending thread, and the sending thread determines the target process corresponding to the target message; The sending thread determines whether the target process is ready. If the target process is ready, the target message is written into the unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message subject of the target message. The receiving thread performs processing operations corresponding to the message type of the target message based on the message topic of the target message, so as to complete inter-process communication between multiple processes and multiple threads.

[0006] Preferably, determining the message order in the lock-free queue bound to each business thread of the sending process includes: For each business thread of the sending process, obtain the business data generated by the business thread, obtain the message topic determined by the business thread based on the business logic of the business data, obtain the message type determined by the sending interface called by the business thread based on the business data, and obtain the global sequence number assigned to the business data by the global sequence number generator; The business data, the message topic, the message type, and the global sequence number are encapsulated to obtain the message corresponding to the business thread. The business thread performs an enqueue operation based on the message and the lock-free queue bound to the business thread itself, and obtains the message sorted in the lock-free queue.

[0007] Preferably, determining the target message based on the messages sorted in all the lock-free queues includes: The message at the head of each of the lock-free queues is determined based on the messages ordered in all the lock-free queues; The message with the smallest global sequence number among all the messages at the head of the queue is identified as the target message.

[0008] Preferably, the sending thread determines the target process corresponding to the target message, including: The sending thread performs a lookup operation based on a preset routing mapping table and the message topic of the target message to obtain the target process corresponding to the target message.

[0009] Preferably, the receiving thread performs processing operations corresponding to the message type of the target message based on the message topic of the target message, to complete inter-process communication involving multiple processes and threads, including: If the message type of the target message is a synchronous request, the business thread corresponding to the target message is paused. The receiving thread performs a task binding operation based on the target message and the dedicated waiting thread of the target process to obtain the message context to be processed by the dedicated waiting thread. The dedicated waiting thread performs business processing and reply operations based on the message context and the message topic of the target message to complete inter-process communication of multiple processes and threads. The message context includes: the sending process, the sending thread, the target message, and the address of the inter-process shared memory region. If the message type of the target message is an asynchronous request, the receiving thread performs a thread allocation operation based on the idle state of the dynamic thread pool of the target message and the target process to obtain an allocated business processing thread. The allocated business processing thread finds the business processing function corresponding to the message topic of the target message, and processes the business data of the target message according to the business processing function to obtain the business processing result, thereby completing the inter-process communication of multiple processes and threads.

[0010] Preferably, the dedicated waiting thread performs business processing and response operations based on the message context, including: The dedicated waiting thread searches for the business processing function corresponding to the message topic of the target message based on the message topic of the target message; The dedicated waiting thread processes the business data of the target message according to the business processing function to obtain the business processing result; The business processing result and the global sequence number of the target message are encapsulated into a reply message. The reply message is sent to the sending process through the address of the inter-process shared memory region. The sending process wakes up the business thread corresponding to the target message in the reply message. The business thread corresponding to the target message continues to execute the preset operation according to the business processing result of the reply message.

[0011] Preferably, the sending thread determines whether the target process is ready by: The sending thread queries the connection status table of the target process to obtain the current connection status of the target process. If the current connection status is ready, then the target process is ready. If the current connection state is not ready, the sending thread writes the target message to its local persistent storage; The daemon thread of the sending process continuously monitors the current connection status of the target process until the target process is ready.

[0012] According to another aspect of the present invention, a multi-process, multi-threaded inter-process communication device is provided, the device comprising: The sorting module is used to determine the order of messages in the lock-free queue bound to each business thread of the sending process. The dequeue module is used by the sending thread in the sending process to determine the target message based on the messages sorted in all the lock-free queues, perform a dequeue operation, and store the target message in the working buffer of the sending thread. The sending thread determines the target process corresponding to the target message. The determining module is used by the sending thread to determine whether the target process is ready. If the target process is ready, the target message is written into a unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message subject of the target message. The execution module is used by the receiving thread to perform processing operations corresponding to the message type of the target message according to the message topic of the target message, so as to complete inter-process communication between multiple processes and multiple threads.

[0013] According to another aspect of the present invention, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the above-described multi-process, multi-threaded inter-process communication method.

[0014] According to another aspect of the present invention, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described multi-process, multi-threaded inter-process communication method.

[0015] By employing the above technical solutions, this invention provides a multi-process, multi-threaded inter-process communication method and apparatus. Through the use of lock-free queues and shared memory regions, communication efficiency is improved. Combined with strict order preservation using global sequence numbers, data order consistency is guaranteed. By monitoring whether the target process is ready and performing processing operations corresponding to the message type of the target message, multi-process, multi-threaded inter-process communication is completed, demonstrating the reliability of stable system operation in high-concurrency scenarios such as network full-traffic threat analysis.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of this application. In the drawings: Figure 1 The diagram illustrates a flowchart of a multi-process, multi-threaded inter-process communication method provided by an embodiment of the present invention. Figure 2A flowchart illustrating another inter-process communication method with multiple processes and threads provided in an embodiment of the present invention is shown. Figure 3 The diagram shows a structural schematic of an inter-process communication device with multiple processes and multiple threads provided in an embodiment of the present invention. Figure 4 This diagram illustrates the structure of another multi-process, multi-threaded inter-process communication device provided in an embodiment of the present invention. Detailed Implementation

[0018] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0019] This embodiment provides a method for inter-process communication in a multi-process, multi-threaded environment, such as... Figure 1 As shown, the method includes: 101. Determine the order of messages in the lock-free queue bound to each business thread of the sending process.

[0020] 102. The sending thread in the sending process determines the target message based on the messages sorted in all the lock-free queues, performs a dequeue operation, stores the target message in the working buffer of the sending thread, and the sending thread determines the target process corresponding to the target message.

[0021] 103. The sending thread determines whether the target process is ready. If the target process is ready, the target message is written into the unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message subject of the target message.

[0022] 104. The receiving thread performs processing operations corresponding to the message type of the target message according to the message topic of the target message, so as to complete inter-process communication of multiple processes and threads.

[0023] Regarding steps 101-104 of the embodiment: On the one hand, the sending of messages from different threads is sorted by a lock-free queue, that is, cross-thread messages are strictly ordered and will not be out of order due to different blocking conditions, and the delivery is orderly, improving the efficiency of enqueueing and dequeueing. On the other hand, the sending thread determines whether the target process is ready, and when the process crashes or starts an interface without registering, the message is guaranteed not to be lost by mistake, thus realizing the reliability of stable system operation in high-concurrency scenarios such as network full traffic threat analysis.

[0024] This invention provides a method and apparatus for inter-process communication in a multi-process and multi-threaded manner. By employing lock-free queues and shared memory regions, the technical solution of this invention improves communication efficiency. Combined with strict order preservation using global sequence numbers, it ensures the consistency of data order. By monitoring whether the target process is ready and performing processing operations corresponding to the message type of the target message, inter-process communication in a multi-process and multi-threaded manner is completed, demonstrating the reliability of stable system operation in high-concurrency scenarios such as network full-traffic threat analysis.

[0025] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, and to fully illustrate the specific implementation process in this embodiment, another inter-process communication method using multiple processes and threads is provided, such as... Figure 2 As shown, the method includes: 201. Determine the order of messages in the lock-free queue bound to each business thread of the sending process.

[0026] In this embodiment, determining the message ordered in the lock-free queue bound to each business thread of the sending process includes: for each business thread of the sending process, obtaining the business data generated by the business thread, obtaining the message topic determined by the business thread based on the business logic of the business data, obtaining the message type determined by the sending interface called by the business thread based on the business data, and obtaining the global sequence number assigned to the business data by the global sequence number generator; performing a message encapsulation operation on the business data, the message topic, the message type, and the global sequence number to obtain the message corresponding to the business thread; and having the business thread perform an enqueue operation based on the message and the lock-free queue bound to the business thread itself to obtain the message ordered in the lock-free queue.

[0027] In a sending process, there is at least one service thread, and each service thread generates service data. For example, sending process A includes service thread A1 and service thread A2. Service thread A1 generates service data 1 at time 1, service thread A2 generates service data 2 at time 1, service thread A1 generates service data 3 at time 2, and so on. The global sequence number is a unique, continuously increasing numerical identifier assigned to each piece of service data. It is used to absolutely determine the order of service data throughout the entire system. For example, if the global sequence number assigned to service data 1 is 0001, the global sequence number assigned to service data 2 is 0002, and the global sequence number assigned to service data 3 is 0003, then the order of service data 1 is before service data 2, and the order of service data 2 is before service data 3.

[0028] Among them, for message topics determined based on the business logic of business data, the message topic is the purpose of the business data. For example, in a threat analysis system, if the business logic of the business data is DDoS attack alert, then the message topic is ddos_attack; if the business logic of the business data is port scan detection, then the message topic is port_scan.

[0029] Specifically, for the message type determined by the sending interface called based on business data, if the called sending interface is a synchronous return interface (supporting direct use of already deserialized responses in question-and-answer scenarios), the message type is a synchronous request; if the called sending interface is an asynchronous callback interface (supporting question-and-answer and multiple-answer scenarios without occupying threads), the message type is an asynchronous request; and if the called sending interface is a send-without-waiting interface, the message type is an asynchronous request. The sending interface supports multiple types and is easy to use. For example, for business data 1, the business data 1 generated by business thread A1, the corresponding message topic, the corresponding message type, and the corresponding global sequence number 0001 are encapsulated to obtain message 1 corresponding to business thread A1 at time 1. Each business thread is bound to a lock-free queue, meaning there is a one-to-one correspondence between business threads and lock-free queues. Message 1 is enqueued, and written into the lock-free queue 1 bound to business thread A1. For example, for business data 2, the message topic, message type, and global sequence number 0002 corresponding to business data 2 generated by business thread A2 are encapsulated to obtain message 2 corresponding to business thread A2 at time 1. Message 2 is then enqueued and written into the lock-free queue 2 bound to business thread A2. Similarly, for business data 3, the message topic, message type, and global sequence number 0003 corresponding to business data 3 generated by business thread A1 are encapsulated to obtain message 3 corresponding to business thread A1 at time 2. Message 3 is then enqueued and written into the lock-free queue 1 bound to business thread A1.

[0030] Among them, message encapsulation involves encoding business data, message topic, message type, and global sequence number to reduce the amount of data transmitted over the network and improve transmission speed.

[0031] 202. The sending thread in the sending process determines the target message based on the messages sorted in all the lock-free queues, performs a dequeue operation, stores the target message in the working buffer of the sending thread, and the sending thread determines the target process corresponding to the target message.

[0032] In this embodiment, determining the target message based on the messages sorted in all the lock-free queues includes: determining the message at the head of each lock-free queue based on the messages sorted in all the lock-free queues; and determining the message with the smallest global sequence number among all the messages at the head of the queues as the target message.

[0033] The sending process A also includes a sending thread, for example, sending thread A3. For each lock-free queue, the message at the head of the queue has the smallest global sequence number. The message at the head of each lock-free queue is retrieved. Since each message at the head of a queue corresponds to a global sequence number, the message with the smallest global sequence number among all the head messages is determined as the target message. For example, lock-free queue 1 has messages 1 and 3 written sequentially, and lock-free queue 2 has message 2 written sequentially. The message at the head of lock-free queue 1 is message 1, and the message at the head of lock-free queue 2 is message 2. The global sequence number of message 1 is 0001, and the global sequence number of message 2 is 0002. The global sequence number of message 1 is less than that of message 2; therefore, message 1 is selected as the target message.

[0034] The sending thread determines the target process corresponding to the target message by performing a lookup operation based on a preset routing mapping table and the message topic of the target message to obtain the target process corresponding to the target message.

[0035] The pre-defined routing table maps processes to message topics. For example, the message topic ddos_attack corresponds to process B, and the message topic port_scan corresponds to process C. Thus, if the target message's topic is ddos_attack, then process B is the target process corresponding to that message.

[0036] 203. The sending thread queries the connection status table of the target process to obtain the current connection status of the target process. If the current connection status is ready, the target process is ready. If the current connection status is not ready, the sending thread writes the target message to its local persistent storage. The daemon thread of the sending process continuously monitors the current connection status of the target process until the target process is ready.

[0037] In this embodiment, the sending process A also includes a daemon thread A4, which continuously monitors the current connection status of the target process by monitoring the socket.

[0038] The sending thread writes the target message to its local persistent storage, which means that the sending thread writes the target message to the sending thread's local persistent storage, waits for the target process to be ready, and then reads the target message from the local persistent storage to perform the target process ready operation in step 204 of the embodiment.

[0039] Local persistent storage ensures that messages are not lost in abnormal situations, thereby guaranteeing high system availability and improving system stability under abnormal conditions.

[0040] 204. If the target process is ready, the target message is written into the unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message subject of the target message.

[0041] In this embodiment, if the target process is ready, the sending process writes the target message into the unique inter-process shared memory area between the sending process and the target process.

[0042] For example, if the target process is process B, we can identify the target process as target process B, and the receiving thread of the target process can be identified as B1.

[0043] In this way, the shared memory area between processes reduces the overhead of data copying and synchronization, thereby reducing communication latency and improving communication efficiency.

[0044] 205. The receiving thread performs processing operations corresponding to the message type of the target message according to the message topic of the target message, so as to complete inter-process communication of multiple processes and threads.

[0045] In this embodiment, the receiving thread performs processing operations corresponding to the message type of the target message based on the message topic of the target message to complete inter-process communication of multiple processes and threads. This includes: if the message type of the target message is a synchronous request, pausing the business thread corresponding to the target message; the receiving thread performs a task binding operation based on the target message and the dedicated waiting thread of the target process to obtain a message context processed by the dedicated waiting thread; and the dedicated waiting thread performs business processing and response operations based on the message context and the message topic of the target message to complete inter-process communication of multiple processes and threads. The message context includes: the sending process, the sending thread, the target message, and the address of the inter-process shared memory region. If the message type of the target message is an asynchronous request, the receiving thread performs a thread allocation operation based on the idle state of the dynamic thread pool of the target message and the target process to obtain an allocated business processing thread. The allocated business processing thread finds the business processing function corresponding to the message topic of the target message, and processes the business data of the target message according to the business processing function to obtain the business processing result, thereby completing the inter-process communication of multiple processes and threads.

[0046] The dedicated waiting thread performs business processing and response operations based on the message context, including: the dedicated waiting thread finding the business processing function corresponding to the message topic of the target message; the dedicated waiting thread processing the business data of the target message according to the business processing function to obtain a business processing result; encapsulating the business processing result and the global sequence number of the target message into a response message, and sending the response message to the sending process through the address of the inter-process shared memory region; the sending process waking up the business thread corresponding to the target message in the response message according to the response message; and the business thread corresponding to the target message continuing to execute preset operations according to the business processing result of the response message.

[0047] For synchronous requests, the business thread corresponding to the target message is paused (after the pause point, the business processing result is used to execute the preset operation). In other words, the business thread corresponding to the target message is suspended and waits to be awakened. After being awakened, the business processing result is obtained, the business thread is resumed, and the business processing result is used to execute the preset operation.

[0048] For example, following steps 201, 202, and 204 of the embodiment, message 1 is the target message, corresponding to business thread A1. Business thread A1 generates business data 1 (business data 1 is, for example, {ip: "192.168.1.100"}), the global sequence number of business data 1 is 0001, and the message type of the target message is a synchronous request. Business thread A1 is paused, and the receiving thread B1 of the target process transfers the target message to a dedicated waiting thread B5. The business processing function corresponding to the message topic of the target message (e.g., ip_blacklist_query) is queried, for example, query_blacklist(), and the following is executed: query_blacklist("192.168.1.100"), resulting in the business processing result {in_blacklist: true, threat_level:"high risk"}. The reply message is then encapsulated with the following global sequence number: 0001, and the business processing result {in_blacklist: true, threat_level: "High risk"}, sending thread A3 receives a reply message, locks business thread A1 by global sequence number: 0001, wakes up business thread A1, and business thread A1 continues to execute the preset operation: if (in_blacklist) {immediately block all traffic of this IP()}.

[0049] For example, following steps 201, 202, and 204 of the embodiment, message 1 is the target message, which corresponds to business thread A1. Business thread A1 generates business data 1 (business data 1 is, for example, {attack_type: "SYN_Flood", target_ip: "10.0.0.1"}). The global sequence number of business data 1 is 0001. The message type of the target message is an asynchronous request. Business thread A1 continues to execute without pausing or waiting. The receiving thread B1 of the target process is assigned a business processing thread B2. The assigned business processing thread B2 checks the message topic of the target message 1 (message topic is, for example, ddos_attack_log) and the corresponding business processing function, such as log_ddos_attack(). It uses log_ddos_attack() to process {attack_type: "SYN_Flood", target_ip: "10.0.0.1"} and obtains the business processing result: log writing successful. At this time, the assigned business processing thread B2 returns to the dynamic thread pool.

[0050] For steps 201-205 of the embodiment, the number of processes and threads can be scaled up, making it suitable for large-scale distributed systems.

[0051] (1) Deep coupling between interface semantics and business scenarios The four calling modes—synchronous return, synchronous callback, asynchronous callback, and send without waiting—are abstracted into general semantics independent of business logic. This allows for free switching between one-to-one, one-to-many, and one-way notifications. Interface behaviors can be dynamically registered and hot-updated at runtime. Upper-layer business logic can extend to new interaction modes without modifying the code, significantly reducing development and maintenance costs.

[0052] (2) Strictly maintain the order of cross-thread messages Centered on a single-producer, single-consumer lock-free queue, the writing to the queue is decoupled from the actual sending: any thread can return after writing to the queue, and a dedicated asynchronous thread executes network sending in a unified manner according to the queue order. Even if the blocking levels of each socket are different, it can ensure that messages arrive at the other end in the order of their generation numbers, eliminating the risk of timing disorder commonly found in traditional multi-threaded environments.

[0053] (3) Zero-loss mechanism when the peer (target process) is unreachable The system continuously monitors the current connection status of the peer; if the peer is not started or has not completed service registration, the message is automatically transferred to local persistent storage, and the daemon thread immediately resends the message after confirming that the peer is ready. The entire process is transparent to the application, ensuring that messages are not lost, duplicated, or disordered without the need for retry code.

[0054] (4) Fault self-healing and resource isolation When a target process exits abnormally, the framework immediately marks its message queue as isolated. New messages are directly rejected with explicit error codes to avoid invalid backlog. Cached messages are transferred to the dead-letter zone, which is stored in shards based on message topic and date, supporting manual replay or batch export. Simultaneously, a resource reclamation thread is triggered to gradually release related handles, memory, and thread resources, preventing a single point of failure from escalating into a system-wide avalanche. Combined with a dynamic thread pool and adaptive rate limiting, overall throughput remains stable even in high-concurrency scenarios.

[0055] This invention provides a method and apparatus for inter-process communication in a multi-process and multi-threaded manner. By employing lock-free queues and shared memory regions, the technical solution of this invention improves communication efficiency. Combined with strict order preservation using global sequence numbers, it ensures the consistency of data order. By monitoring whether the target process is ready and performing processing operations corresponding to the message type of the target message, inter-process communication in a multi-process and multi-threaded manner is completed, demonstrating the reliability of stable system operation in high-concurrency scenarios such as network full-traffic threat analysis.

[0056] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this invention provides a multi-process, multi-threaded inter-process communication device, such as... Figure 3 As shown, the device includes: a sorting module 31, a dequeue module 32, a determination module 33, and an execution module 34; Sorting module 31 is used to determine the order of messages in the lock-free queue bound to each business thread of the sending process; The dequeue module 32 is used by the sending thread in the sending process to determine the target message based on the messages sorted in all the lock-free queues, perform a dequeue operation, and store the target message in the working buffer of the sending thread. The sending thread determines the target process corresponding to the target message. The determining module 33 is used by the sending thread to determine whether the target process is ready. If the target process is ready, the target message is written into a unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message topic of the target message. The execution module 34 is used by the receiving thread to perform processing operations corresponding to the message type of the target message according to the message topic of the target message, so as to complete inter-process communication between multiple processes and multiple threads.

[0057] Accordingly, in order to determine the order of messages in the lock-free queue bound to each business thread of the sending process, the sorting module 31 is specifically used to: obtain the business data generated by the business thread, obtain the message topic determined by the business thread based on the business logic of the business data, obtain the message type determined by the sending interface called by the business thread based on the business data, and obtain the global sequence number assigned to the business data by the global sequence number generator; perform message encapsulation operation on the business data, the message topic, the message type, and the global sequence number to obtain the message corresponding to the business thread; and perform an enqueue operation on the message according to the lock-free queue bound to the business thread itself to obtain the message ordered in the lock-free queue.

[0058] Accordingly, in order to determine the target message based on the messages sorted in all the lock-free queues, the dequeue module 32 is specifically used to determine the message at the head of each lock-free queue based on the messages sorted in all the lock-free queues; and to determine the message with the smallest global sequence number among all the messages at the head of the queues as the target message.

[0059] Accordingly, in order for the sending thread to determine the target process corresponding to the target message, the dequeue module 32 is specifically used for the sending thread to perform a lookup operation based on a preset routing mapping table and the message topic of the target message to obtain the target process corresponding to the target message.

[0060] Accordingly, in order for the receiving thread to perform processing operations corresponding to the message type of the target message according to the message topic of the target message, so as to complete inter-process communication of multi-process and multi-threaded processes, the execution module 34 includes: a first execution unit 341 and a second execution unit 342. The first execution unit 341 is specifically used to pause the business thread corresponding to the target message if the message type of the target message is a synchronous request. The receiving thread performs a task binding operation based on the target message and the dedicated waiting thread of the target process to obtain the message context to be processed by the dedicated waiting thread. The dedicated waiting thread performs business processing and reply operations based on the message context and the message topic of the target message to complete inter-process communication of multiple processes and threads. The message context includes: the sending process, the sending thread, the target message, and the address of the inter-process shared memory region. The second execution unit 342 is specifically used to, if the message type of the target message is an asynchronous request, perform a thread allocation operation based on the idle state of the dynamic thread pool of the target message and the target process to obtain an allocated business processing thread. The allocated business processing thread finds the business processing function corresponding to the message topic of the target message based on the message topic of the target message. The allocated business processing thread processes the business data of the target message according to the business processing function to obtain the business processing result, thereby completing inter-process communication of multiple processes and multiple threads.

[0061] Accordingly, in order for the dedicated waiting thread to perform business processing and reply operations based on the message context, the first execution unit 341 is specifically used for the dedicated waiting thread to find the business processing function corresponding to the message topic of the target message based on the message topic of the target message; the dedicated waiting thread to process the business data of the target message according to the business processing function to obtain the business processing result; to encapsulate the business processing result and the global sequence number of the target message into a reply message, and to send the reply message to the sending process through the address of the inter-process shared memory area; the sending process to wake up the business thread corresponding to the target message in the reply message according to the reply message; and the business thread corresponding to the target message to continue to execute the preset operation according to the business processing result of the reply message.

[0062] Accordingly, in order for the sending thread to determine whether the target process is ready, the determining module 33 is specifically used for the sending thread to query the connection status table of the target process to obtain the current connection status of the target process. If the current connection status is ready, then the target process is ready; if the current connection status is not ready, then the sending thread writes the target message to its local persistent storage; the daemon thread of the sending process continuously monitors the current connection status of the target process until the target process is ready.

[0063] It should be noted that other corresponding descriptions of the functional units involved in the multi-process, multi-threaded inter-process communication device provided in this embodiment can be found in [reference]. Figures 1 to 2 The corresponding description will not be repeated here.

[0064] Based on the above, Figures 1 to 2 Accordingly, this embodiment also provides a storage medium, which may be volatile or non-volatile, storing a computer program that, when executed by a processor, implements the above-described method. Figures 1 to 2 The method for inter-process communication in a multi-process, multi-threaded environment is shown.

[0065] Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present invention.

[0066] Based on the above, Figures 1 to 2 The method shown and Figure 3 , Figure 4 To achieve the above objectives, the present application also provides a computer device, specifically a personal computer, server, network device, etc., as shown in the illustrated embodiment. This computer device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 2 The method for inter-process communication in a multi-process, multi-threaded environment is shown.

[0067] Optionally, the computer device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0068] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0069] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned computer device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the non-volatile storage medium, as well as communication with other hardware and software in the information processing entity device.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0071] This invention provides a method and apparatus for inter-process communication in a multi-process and multi-threaded manner. By employing lock-free queues and shared memory regions, the technical solution of this invention improves communication efficiency. Combined with strict order preservation using global sequence numbers, it ensures the consistency of data order. By monitoring whether the target process is ready and performing processing operations corresponding to the message type of the target message, inter-process communication in a multi-process and multi-threaded manner is completed, demonstrating the reliability of stable system operation in high-concurrency scenarios such as network full-traffic threat analysis.

[0072] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or they can be located in one or more apparatuses different from this embodiment, with corresponding changes. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0073] The serial numbers used above are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenarios. The above disclosures are merely a few specific implementation scenarios of the present invention; however, the present invention is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for inter-process communication in a multi-process, multi-threaded environment, characterized in that, The method includes: Determine the order of messages in the lock-free queue bound to each business thread of the sending process; The sending thread in the sending process determines the target message based on the messages sorted in all the lock-free queues, performs a dequeue operation, stores the target message in the working buffer of the sending thread, and the sending thread determines the target process corresponding to the target message; The sending thread determines whether the target process is ready. If the target process is ready, the target message is written into the unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message subject of the target message. The receiving thread performs processing operations corresponding to the message type of the target message based on the message topic of the target message, so as to complete inter-process communication between multiple processes and multiple threads.

2. The method according to claim 1, characterized in that, The message sorting within the lock-free queue bound to each business thread of the sending process includes: For each business thread of the sending process, obtain the business data generated by the business thread, obtain the message topic determined by the business thread based on the business logic of the business data, obtain the message type determined by the sending interface called by the business thread based on the business data, and obtain the global sequence number assigned to the business data by the global sequence number generator; The business data, the message topic, the message type, and the global sequence number are encapsulated to obtain the message corresponding to the business thread. The business thread performs an enqueue operation based on the message and the lock-free queue bound to the business thread itself, and obtains the message sorted in the lock-free queue.

3. The method according to claim 2, characterized in that, The step of determining the target message based on the messages sorted in all the lock-free queues includes: The message at the head of each of the lock-free queues is determined based on the messages ordered in all the lock-free queues; The message with the smallest global sequence number among all the messages at the head of the queue is identified as the target message.

4. The method according to claim 2, characterized in that, The sending thread determines the target process corresponding to the target message, including: The sending thread performs a lookup operation based on a preset routing mapping table and the message topic of the target message to obtain the target process corresponding to the target message.

5. The method according to claim 1, characterized in that, The receiving thread performs processing operations corresponding to the message type of the target message based on the message topic of the target message, in order to complete inter-process communication involving multiple processes and threads, including: If the message type of the target message is a synchronous request, the business thread corresponding to the target message is paused. The receiving thread performs a task binding operation based on the target message and the dedicated waiting thread of the target process to obtain the message context to be processed by the dedicated waiting thread. The dedicated waiting thread performs business processing and reply operations based on the message context and the message topic of the target message to complete inter-process communication of multiple processes and threads. The message context includes: the sending process, the sending thread, the target message, and the address of the inter-process shared memory region. If the message type of the target message is an asynchronous request, the receiving thread performs a thread allocation operation based on the idle state of the dynamic thread pool of the target message and the target process to obtain an allocated business processing thread. The allocated business processing thread finds the business processing function corresponding to the message topic of the target message, and processes the business data of the target message according to the business processing function to obtain the business processing result, thereby completing the inter-process communication of multiple processes and threads.

6. The method according to claim 5, characterized in that, The dedicated waiting thread performs business processing and response operations based on the message context, including: The dedicated waiting thread searches for the business processing function corresponding to the message topic of the target message based on the message topic of the target message; The dedicated waiting thread processes the business data of the target message according to the business processing function to obtain the business processing result; The business processing result and the global sequence number of the target message are encapsulated into a reply message. The reply message is sent to the sending process through the address of the inter-process shared memory region. The sending process wakes up the business thread corresponding to the target message in the reply message. The business thread corresponding to the target message continues to execute the preset operation according to the business processing result of the reply message.

7. The method according to claim 1, characterized in that, The sending thread determines whether the target process is ready, including: The sending thread queries the connection status table of the target process to obtain the current connection status of the target process. If the current connection status is ready, then the target process is ready. If the current connection state is not ready, the sending thread writes the target message to its local persistent storage; The daemon thread of the sending process continuously monitors the current connection status of the target process until the target process is ready.

8. A multi-process, multi-threaded inter-process communication device, characterized in that, The device includes: The sorting module is used to determine the order of messages in the lock-free queue bound to each business thread of the sending process. The dequeue module is used by the sending thread in the sending process to determine the target message based on the messages sorted in all the lock-free queues, perform a dequeue operation, and store the target message in the working buffer of the sending thread. The sending thread determines the target process corresponding to the target message. The determining module is used by the sending thread to determine whether the target process is ready. If the target process is ready, the target message is written into a unique inter-process shared memory area between the sending process and the target process. The target process reads the target message from the inter-process shared memory area into the target process's memory space. The receiving thread of the target process parses the target message to obtain the message type and message subject of the target message. The execution module is used by the receiving thread to perform processing operations corresponding to the message type of the target message according to the message topic of the target message, so as to complete inter-process communication between multiple processes and multiple threads.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the inter-process communication method of any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored on a storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the inter-process communication method of any one of claims 1 to 7.

Images