Message broker invocation methods and their devices, equipment, and media

Abstract

This application relates to a message broker invocation method, apparatus, device, medium, and product thereof. The method includes: loading a message stream from a message queue of a message server into a buffer, the message stream including multiple message bodies; determining the position pointer of the message body in the message stream within the buffer for the current caller based on the current caller's positioning reference information; identifying other callers whose position pointers are synchronized with the current caller, and setting the current caller and other callers to belong to the same synchronization group; when the position pointer of any caller in the synchronization group moves forward, allocating the message body pointed to by that position pointer to a caller in the synchronization group that matches that message body for consumption. By controlling the synchronization of position pointers and utilizing a group mechanism, the method achieves rapid matching of callers who need to read message bodies, improving message body consumption efficiency, saving system overhead, and significantly improving the operational efficiency of online services provided by callers.

Description

Technical Field

[0001] This application relates to network communication technology, and in particular to a message broker invocation method, apparatus, device, and medium thereof. Background Technology

[0002] In the e-commerce field, message queues are widely used to asynchronously process various platform data, such as order data, payment data, and coupon data. Message service systems that provide message queue services often use topics to distinguish message services with different functions. In addition, to reflect the advantages of distribution, message service systems often set up multiple storage partitions for each topic so that messages can be distributed and stored in the message service cluster.

[0003] In the traditional technical architecture, when a large number of callers call the message body in the same message queue, even if different callers use the same topic and the same storage partition for the message queue, their calls to the message body in the message queue are concurrent calls at the granularity of the message body. In this case, even if there are multiple callers synchronously accessing the same message body, they still need to go through the potentially identical message preprocessing process, and the business logic of each caller consuming the same message body is not related to each other.

[0004] Message queue producers typically encapsulate various message bodies according to their own data structures. These messages circulate within the same message queue, and the data objects within each type of message body can be diverse. However, different callers, due to the different online services they provide, may require different sets of data objects, or different message body types, even though they rely on the same producer's message bodies. Therefore, in practice, when a large number of callers concurrently consume message bodies from the same message queue, each caller needs to transmit and process a large amount of redundant data, resulting in significant waste of computational resources at each node in the message body circulation process. In view of this, it is necessary to optimize the traditional message mechanism. Summary of the Invention

[0005] The purpose of this application is to solve the above-mentioned problems by providing a message broker invocation method and corresponding apparatus, devices, non-volatile readable storage media, and computer program products.

[0006] According to one aspect of this application, a message broker invocation method is provided, comprising the following steps:

[0007] A message stream, comprising multiple message bodies, is loaded from the message queue of the message server into the buffer.

[0008] Based on the current caller's positioning reference information, the position pointer of the message body in the message stream in the current caller's buffer is determined.

[0009] Identify other callers whose position pointers are synchronized with the current caller, and set the current caller and other callers to belong to the same synchronization group;

[0010] When the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is assigned to the caller in the synchronization group that matches the message body for consumption.

[0011] In an optional embodiment, before determining the position pointer of the message body in the message stream within the buffer for the current caller, based on the current caller's positioning reference information, the process includes:

[0012] In response to the current caller's registration request, the registration information carried in the registration request is extracted. The registration information includes the positioning reference information, which includes the message topic, topic partition, and message offset.

[0013] Based on the message topic and topic partition, determine whether the source message topic and source topic partition of the message stream in the cache belong to the message topic and topic partition specified in the positioning reference information. If they do not belong, push the topic partition list to the current caller and terminate the response to the registration request.

[0014] If the message topic and topic partition specified in the positioning reference information are included, the registration information of the current caller will be written into the registration information table for future use.

[0015] In an optional embodiment, based on the current caller's positioning reference information, determining the position pointer of the message body in the message stream read from the buffer by the current caller includes:

[0016] The position pointer of the current caller in the message stream of the buffer is determined based on the message topic, topic partition and message offset in the current caller's positioning reference information. The message body pointed to by the position pointer is the current message body.

[0017] In response to a first consumption event where the current message body is consumed by the current caller, the position pointer of the current caller is moved forward relative to the current message body to the next message body in the message stream.

[0018] In an optional embodiment, identifying other callers whose position pointers are synchronized with the current caller, and setting the current caller and other callers to belong to the same synchronization group, includes:

[0019] Determine if the current caller's position pointer is the same as that of any other caller. If they are the same, set the current caller and that other caller to belong to the same current synchronization group.

[0020] Determine whether the position pointer of the caller in the current synchronization group is the same as the position pointer of the caller in any other synchronization group. If they are the same, merge the current synchronization group with the other synchronization group into the same synchronization group.

[0021] In an optional embodiment, when the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is assigned to a caller in the synchronization group that matches the message body for consumption, including:

[0022] After any caller in the synchronization group finishes consuming the message body at the position pointer, the position pointer is triggered to move forward to the next message body in the message stream, and the next message body is taken as the target message body;

[0023] The appropriate caller is determined based on the matching rules corresponding to each caller and the caller that matches the target message body.

[0024] The target message body is sent to each of the eligible callers for consumption.

[0025] In an optional embodiment, determining the caller that matches the target message body as a qualified caller according to the matching rules corresponding to each caller includes:

[0026] Obtain the matching rules provided by each caller in the synchronization group through its registration information. The matching rules are used to specify a specific data object and its corresponding eligible data characteristics.

[0027] Based on each caller as the target caller, it is determined whether the attribute data of a specific data object in the target message body matches the qualified data characteristics of the target caller. When they match, the target caller is determined to be a qualified caller of the target message body.

[0028] In an optional embodiment, determining whether the attribute data of a specific data object in the target message body matches the eligibility data characteristics of the target caller, and determining the target caller as an eligibility caller of the target message body when they match, includes:

[0029] A sliding window of preset byte length slides across the serialized data corresponding to the target message body in the buffer, and the serialized data corresponding to the sliding window is read out.

[0030] The serialized data corresponding to the sliding window is deserialized to obtain object data, which includes multiple data objects and their corresponding attribute data.

[0031] Identify whether the object data contains a specific data object specified in the matching rules of the target caller, and whether the attribute data of the contained specific data object matches the eligible data features specified in the matching rules. When a match is found, determine that the target caller is an eligible caller of the target message body.

[0032] If a mismatch occurs, the sliding window continues to iterate and identify until the serialized data of the target message body is identified.

[0033] According to another aspect of this application, a message broker invocation apparatus is provided, comprising:

[0034] The message loading module is configured to load a message stream from the message queue of the message server into the buffer, wherein the message stream includes multiple message bodies;

[0035] The pointer tracking module is configured to determine the position pointer of the message body in the message stream within the buffer area for the current caller, based on the current caller's positioning reference information.

[0036] The group maintenance module is configured to identify other callers whose position pointers are synchronized with the current caller, and set the current caller and other callers to belong to the same synchronization group;

[0037] The matching consumption module, when the position pointer of any caller in the synchronization group moves forward, assigns the message body pointed to by the position pointer to the caller in the synchronization group that matches the message body for consumption.

[0038] According to another aspect of this application, a message broker invocation device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke a computer program stored in the memory to perform the steps of the message broker invocation method described in this application.

[0039] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the message broker invocation method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, performs the steps included in the method.

[0040] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0041] Compared to existing technologies, this application embeds a proxy mechanism between the caller and the message queue. First, the message stream from the message queue is loaded into a buffer. For each caller, based on the position pointer of the message body in the message stream, other callers with synchronized position pointers are found. All callers with synchronized position pointers are constructed into the same synchronization group. Then, whenever the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is synchronously distributed to the caller in the same synchronization group that matches that message body for consumption. Thus, by using the synchronization group mechanism to quickly match callers who need to read the message body based on whether the position pointers are synchronized, unnecessary message bodies are filtered for the caller, eliminating the hassle of individual message body preprocessing and identification by each caller in the same synchronization group. This can improve message body consumption efficiency in high-concurrency call scenarios and save unnecessary computer system overhead. For various application scenarios in e-commerce platforms that require asynchronous message processing, it can significantly improve the operating efficiency of online services provided by the caller and optimize the user experience of these online services. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of the business logic architecture of the message service system in this application;

[0044] Figure 2 This is a schematic diagram of the network architecture of an exemplary application environment for this application;

[0045] Figure 3 This is a flowchart illustrating one embodiment of the message broker invocation method of this application;

[0046] Figure 4 This is a schematic diagram of the business logic architecture of the proxy server in this application;

[0047] Figure 5 This is a flowchart illustrating the registration process of the current caller in this embodiment of the application;

[0048] Figure 6 This is a schematic diagram of the process of distributing the target message body based on a synchronous group in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the process of matching the eligible caller in the synchronization group based on the target message body in the embodiments of this application;

[0050] Figure 8 This is a schematic diagram of the message broker invocation device of this application;

[0051] Figure 9 This is a schematic diagram of the structure of a message broker invocation device used in this application. Detailed Implementation

[0052] E-commerce platforms widely use message service systems to support their business operations. These systems are distributed across message servers in a message service cluster. Each message server maintains a message queue and provides corresponding message services to various online services on the e-commerce platform. This enables these online services to perform asynchronous coordination functions such as order placement, payment retries, and coupon distribution.

[0053] Figure 1 The business logic architecture of the messaging service system of this application is shown, such as... Figure 1 As shown, the message service system is based on a topic subscription mechanism, serving multiple message topics and providing message queue services to callers subscribing to the corresponding message topics. Logically, each message topic can have multiple topic partitions. Each topic partition provides a corresponding message queue for storing and transmitting message bodies generated by the message producer. These topic partitions can provide services in parallel to achieve load balancing for the same message topic. The message service system manages each message topic on an instance basis. A message server in the message service cluster can be used to deploy one or more instances. For ease of understanding, consider the scenario where a single message server maintains only a single instance. It's easy to see that each instance can accommodate multiple message topics. Because the message service system sets up replicas for each topic partition of each message topic and distributes these replicas across instances, the same instance can accommodate different topic partitions of the same message topic. However, only one topic partition belonging to the original can be open at a time; the others, belonging to the same message topic, maintain their replica status. Within each topic partition, the message bodies written by the producer are maintained in an orderly manner according to the queue mechanism, and the storage location of each message body is determined by the message offset.

[0054] Accordingly, the message service system is responsible for maintaining the topic partition list of the entire system, so as to facilitate the determination of the corresponding message queue by message topic and topic partition, locate the corresponding message queue for various online service callers as message consumers, and provide the message body in the corresponding message queue.

[0055] like Figure 2In the network architecture shown, the messaging service system 80 can be deployed on the Internet together with the e-commerce platform 82 to provide corresponding services to users on the e-commerce platform 82. In some embodiments, the messaging service system 80 can be jointly deployed with the e-commerce platform 82 in the same service cluster, so that the messaging service system 80 constitutes a part of the associated services of the e-commerce platform 82, so that the messaging service system 80 can share the infrastructure of the e-commerce platform 82 to operate and provide services.

[0056] The messaging service system 80 can act as middleware to provide messaging services to the e-commerce platform 82, participating in the operation of various online services implemented within the e-commerce platform 82 and ensuring the normal operation of these services. Each online service within the e-commerce platform can be both a message producer and a message consumer. Message producers can generate corresponding message bodies for specific message topics, while message consumers can retrieve the message bodies generated for a specified message topic. For example, the first online service corresponding to a product order, acting as a message producer, generates a message body for an order awaiting payment, which can be invoked by the second online service corresponding to the order payment process to complete the order payment.

[0057] An exemplary e-commerce platform 82 provides supply and demand matching of products and / or services to the general public through the Internet infrastructure. In e-commerce platform 82, products and / or services are provided as commodity information. For the sake of simplicity, the concepts of commodity and product are used in this application to refer to the products and / or services in e-commerce platform 82. Specifically, these may be physical products, digital products, tickets, service subscriptions, other offline services, etc.

[0058] In reality, various entities can access e-commerce platform 82 as users and utilize its online services to participate in the business activities facilitated by the platform. These entities can be natural persons, legal persons, or social organizations. Corresponding to the two types of entities in business activities—merchants and consumers—e-commerce platform 82 has two corresponding categories of users: merchant users and consumer users. Entities involved in the product distribution chain in business activities, including manufacturers, sellers, retailers, and logistics providers, can all use online services on e-commerce platform 82 as merchant users. Similarly, consumers in business activities, including actual or potential consumers, can use online services on e-commerce platform 82 as consumer users. In actual business activities, the same entity can operate as both a merchant user and a consumer user; this should be interpreted flexibly.

[0059] The infrastructure used to deploy the e-commerce platform 82 and the messaging service system 80 mainly includes the backend architecture and frontend devices. The backend architecture runs various online services through a service cluster, including middleware or frontend services for the platform, services for consumers, and services for merchants, to enrich and improve its service functions. The frontend devices mainly cover the terminal devices used by users as clients to access the e-commerce platform 82 and / or the fourth-party logistics system 80, including but not limited to various mobile terminals, personal computers, and point-of-sale devices. For example, consumer users can access the webpage of an online store implemented by the e-commerce platform 82 through their terminal devices, trigger the shopping process by clicking the shopping button provided on the webpage, and call various online services provided by the e-commerce platform 82 during the shopping process to achieve the purpose of placing an order.

[0060] In some embodiments, the e-commerce platform 82 may be implemented via a processing facility including a processor and memory, which stores a set of instructions that, when executed, cause the e-commerce platform 82 to perform the e-commerce and support functions as described in this application. The processing facility may be part of a server, client, network infrastructure, mobile computing platform, cloud computing platform, fixed computing platform, or other computing platform, and may provide electronic components, merchant devices, payment gateways, application developers, marketing channels, transportation providers, customer devices, point-of-sale devices, etc., for the e-commerce platform 82.

[0061] The e-commerce platform 82 and the messaging service system 80 can provide online services such as cloud computing services, Software as a Service (SaaS), Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Desktop as a Service (DaaS), Managed Software as a Service, Mobile Backend as a Service (MBaaS), and Information Technology Management as a Service (ITMaaS). In some embodiments, the various functional components of the e-commerce platform 82 and the messaging service system 80 can be implemented to operate on various platforms and operating systems. For example, for an online store, its administrator user enjoys the same or similar functions regardless of whether it is on iOS, Android, HomonyOS, or a web page.

[0062] E-commerce platform 82 enables merchants to create their own independent websites to run their online stores. It provides merchants with corresponding business management engine instances for establishing, maintaining, and operating one or more online stores across these independent websites. The business management engine instance can be used for content management, task automation, and data management of one or more online stores. It can be configured through interfaces or built-in components to support various specific business processes in the online store. Independent websites are the infrastructure of e-commerce platform 82, which has cross-border service capabilities. Merchants can maintain their online stores relatively independently and centrally based on these independent websites. Independent websites typically have dedicated domain names and storage space, and different independent websites are relatively independent. E-commerce platform 82 can provide standardized or customized technical support for a large number of independent websites, allowing merchants to customize their own suitable business management engine instances and use these instances to maintain their one or more online stores. The message service system 80 provides corresponding message services for the operation of these online stores.

[0063] The messaging service system 80 can pre-configure corresponding interface functions within the business management engine of merchant users. By using these interface functions, merchant users can better utilize the messaging services provided by the messaging service system 80 to implement complete business activities. For example, a merchant user can batch-aggregate a large number of product orders generated from one or more online stores operating on one or more e-commerce platforms 82 through the function pages provided by the business management engine, construct a logistics waybill list, and then transmit the corresponding message body through the messaging service system to notify the logistics system to initiate the specific logistics waybill processing process.

[0064] Online stores can be configured and maintained in the backend by merchant users logging into their Business Management Engine instance as administrators. Supported by the various online services provided by the e-commerce platform 82's infrastructure, merchant users can configure various functions within their online stores and view various data as administrators. For example, merchant users can manage various aspects of their online stores, such as viewing recent online store activities, updating the online store's product catalog, managing orders, recent visit activity, and total order activity. Merchant users can also view more detailed information about their business and visitors to their online store by obtaining reports or metrics, such as displaying a sales summary of the merchant's overall business, specific sales and engagement data from promotional sales and marketing channels, etc.

[0065] E-commerce platform 82 can provide communication facilities and associated merchant interfaces for electronic communication and marketing. For example, it can utilize electronic messaging aggregation facilities to collect and analyze communication interactions between merchants, consumers, merchant devices, customer devices, point-of-sale devices, etc., aggregating and analyzing communications to increase the potential for product sales. For instance, a consumer may have product-related questions, potentially leading to a dialogue between the consumer and the merchant (or an automated processor-based agent representing the merchant). The communication facilities are responsible for the interaction and providing the merchant with analysis on how to increase the probability of a sale. During this process, the messaging service system 80 can still function as a notification service, facilitating the dialogue between consumers and merchants.

[0066] In some embodiments, applications suitable for installation on terminal devices can be provided to serve the access needs of different users. This allows various users to access the e-commerce platform 82 and / or the message service system 80 directly or indirectly to use message service functions, such as the merchant backend module of an online store in the e-commerce platform 82, by running the application on their terminal devices. During the process of realizing business activities through these functions, the message service system 80 can implement various functions related to business activities as middleware or online services and expose corresponding interfaces. Then, the toolkits corresponding to the interface access functions are embedded into the application to achieve function expansion and task implementation. The business management engine can include the basic functions of the message service system 80 and expose these functions to online services and / or application calls via APIs. Online services and applications provide or call message bodies corresponding to various message topics by remotely calling the corresponding APIs.

[0067] With the support of various components of the Business Management Engine instance, the e-commerce platform 82 can provide online shopping functionality, enabling merchants to connect with customers in a flexible and transparent manner. Consumers can select items online, create orders, provide delivery addresses in the orders, and confirm payment. Merchants can then review and complete or cancel orders. The review component included with the Business Management Engine instance ensures compliant use of business processes, guaranteeing that orders are suitable for fulfillment before actual execution. Orders may sometimes be fraudulent and require verification (e.g., ID checks). Payment methods that require merchants to wait for receipt of funds can mitigate this risk. Order risks may arise from fraud detection tools submitted by third parties through order risk APIs, etc. Before fulfillment, merchants may need to obtain or wait to receive payment information to mark the order as paid before preparing to deliver the product. Such situations can all be reviewed accordingly. The review process can be implemented by the fulfillment component. Merchants can leverage fulfillment components to review and adjust operations, and trigger related fulfillment services. These include: manual fulfillment services, used when merchants select and pack products into boxes, purchase shipping labels and enter tracking numbers, or simply mark items as fulfilled; custom fulfillment services, which can define email notifications; API fulfillment services, which can trigger third-party applications to create fulfillment records; legacy fulfillment services, which can trigger custom API calls from the Commerce Management Engine to third parties; and gift card fulfillment services, which can generate and activate gift cards. Merchants can use an order printer application to print shipping documents. The fulfillment process can be executed once items are packed and ready for shipment, tracked, delivered, and verified by the consumer.

[0068] It can be seen that the messaging service system can play the role of intermediate data notification and transmission in various business links of the e-commerce platform, and is an important functional component to ensure the stable operation of the e-commerce platform.

[0069] Please see Figure 3 According to a message broker invocation method provided in this application, in one embodiment, it includes the following steps:

[0070] Step S5100: Load a message stream from the message queue of the message server into the buffer area, wherein the message stream includes multiple message bodies;

[0071] Figure 4This is a schematic diagram of the logical architecture of the message broker module embedded in the message service system in this application. The message broker module is responsible for the registration and message invocation services of the message consumer, invoking the appropriate message body for the message consumer acting as the invoker. The message broker module of this application can be implemented as a computer program product and deployed to run in the message service system. For example, it can be deployed to a message server in the message service system with one or more instances allocated, to proxy the invocation of message topics and message queues of each topic partition in one or more instances, providing message invocation services to each invoker acting as a message consumer. It should be noted that in other embodiments, the message broker module of this application can also be deployed to run on a separate server, as long as it is suitable for communicating with the message server to access its messages. Regardless of whether the message broker module relies on a standalone server or is embedded in a message server, since it has the message broker function of this application, a server equipped with a message broker module essentially constitutes a proxy server, and its function module for proxying and calling relevant messages constitutes the corresponding middleware. It can open service interfaces for various related online services in the e-commerce platform, so that these online services can act as callers and use the message broker module in the proxy server to provide them with messages corresponding to specific message topics and specific topic partitions.

[0072] The message broker module of this application can stream the message stream generated in the message queue it is responsible for brokering to a cache area. Alternatively, when the message broker module is running on the message server of the message service system, the storage space corresponding to the topic partition of the message server used to store the message queue can be regarded as a cache area. The message broker module can identify and process the message body based on this cache area so as to provide it to the corresponding callers.

[0073] In the message server, each topic partition's corresponding storage space is named and stored in the form of folders. The message bodies in the message stream are sequentially encoded and divided into multiple fragments according to the encoding space. These message bodies are stored in fragments. Since each topic partition has a starting code, and the sequentially encoded message bodies have their corresponding codes, the message body's code can be regarded as the message body's storage address. The target message body is determined using the corresponding message offset. Of course, the corresponding mechanism can also be directly mapped to the cache area, thereby determining the corresponding target message body in the cache area.

[0074] When providing its proxy invocation service, the message broker module takes into account that the basic unit for maintaining message bodies in the message service system is the message queue, and the message queue carries multiple message bodies in a streaming manner. Therefore, it can also execute each step of this application based on each message queue. This makes it convenient and efficient to implement the message broker module of this application as a standardized computer program product, which constitutes a service middleware when the computer program product is running.

[0075] Step S5200: Based on the current caller's positioning reference information, start determining the position pointer of the message body in the message stream in the current caller's buffer.

[0076] The callers accessing the messaging service system can be third-party programs or similar programs with streaming capabilities. These programs can record, either independently or with the help of the message server, the message offset of the message body they have consumed as callers. Alternatively, these programs can set their own message offset when initializing a message in a message queue. In short, this message offset can be used to locate the initial message body in the message queue it wants to access, serving as a pointer to the current message body being invoked within the message stream.

[0077] To locate the message body in the message queue, each caller can submit a positioning reference information in advance. This positioning reference information can include specific specifications for the instance, message topic, topic partition, and message offset, depending on its protocol with the message broker module of this application. For example, if the message broker module has already specified an instance by default, the positioning reference information can only provide the message topic, topic partition, and message offset; or, if the message broker module has specified the instance and message topic by default, the positioning reference information can also only specify the topic partition and message offset. Therefore, the purpose of the positioning reference information is to locate the starting message body position of the message queue that the caller needs to read and initialize the caller's position pointer in the message queue. The data items included in the positioning reference information and their data format can be flexibly set.

[0078] When the message broker module first receives a call request from a caller, it takes this caller as the current caller, obtains the current caller's location reference information, and then determines the position of the message body of the current caller in the message stream in the buffer based on the location reference information. Thus, the position pointer for reading the message body of the current caller is initialized through this position. Subsequently, as the caller reads the message body in the message queue in a certain order, this position pointer will also be moved forward accordingly.

[0079] In general, the message broker module will respond to multiple caller requests for message proxying simultaneously. In this case, each caller can maintain its corresponding position pointer according to the principles described above. Therefore, it is easy to understand that for the message broker module, each caller will have its own position pointer. In some embodiments, this position pointer can be synchronized between the message broker module and the caller itself to facilitate maintaining consistency between the message broker module and the caller.

[0080] Step S5300: Identify other callers whose position pointers are synchronized with the current caller, and set the current caller and other callers to belong to the same synchronization group;

[0081] Because e-commerce platforms typically run a large number of online services simultaneously, some of these services act as message producers, generating large amounts of message bodies and writing them to the corresponding message queues in the message service system. Others act as message consumers, i.e., callers, accessing message bodies in the message queues. Therefore, even when the message broker module is serving the proxy calls for message bodies in the same message queue, it may serve multiple callers simultaneously. However, the initial positioning reference information provided by different callers accessing message bodies in the same message queue may not be exactly the same, so they may not be synchronized at first. Intriguingly, during the process of multiple callers accessing the same message queue, the access progress of each caller often shows signs of "convergent evolution." That is, each caller will try to continuously advance the consumption of message bodies as quickly as possible, inevitably leading to each caller accessing the latest message body in the message queue. In other words, these different callers accessing the same message queue tend to keep pace, exhibiting synchronized position pointers.

[0082] Considering that each message broker module performs adaptive preprocessing when pushing message bodies to each caller—for example, deserializing message bodies transmitted as binary serialization data in the message queue into object data—if this work were done separately for each caller, as the number of callers increases, the message broker module would have to handle a large amount of repetitive work, leading to unnecessary computer system overhead and affecting the throughput of the broker server. Therefore, this application identifies whether the position pointers of callers accessing the same message queue are synchronized. In one embodiment, this can be done when the current caller updates its position pointer, comparing its position pointer with that of any other caller accessing the same message queue. If they match, all callers with the same position pointer are set to belong to the same synchronization group. If this synchronization group already exists, the current caller only needs to be associated with this group; if it does not exist, a new synchronization group can be created accordingly.

[0083] It should be noted that the synchronization group in this application differs from the traditional groups used in message service systems to distinguish consumers who can only access different topic partitions under the same message topic. Such traditional groups merge multiple callers into one group and control that each caller can only access one topic partition under the same message topic at any given time. In other words, callers in a traditional group are mutually exclusive when accessing topic partitions of the same message topic with other callers in the same group. Conversely, callers in the synchronization group of this application can access message bodies within the same topic partition through the message broker module. Therefore, callers in a synchronization group are allowed to synchronously access message bodies within the same topic partition.

[0084] Based on the above principles, when there are a large number of callers, it is possible that at certain times, there are multiple different synchronization groups, each with a different group of callers. However, if, over time, the position pointers of the callers in different synchronization groups are eventually synchronized, then the synchronization group to which each caller belongs can be flexibly controlled. Alternatively, if the position pointers of all callers in each synchronization group have been synchronized, then the synchronization groups can be merged into the same synchronization group, thereby maintaining the affiliation between the caller and the synchronization group in a timely manner.

[0085] Step S5400: When the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is assigned to the caller in the synchronization group that is compatible with the message body for consumption.

[0086] In one embodiment, in the message broker module, the operation of each caller consuming the message body in the message queue can be performed in its own thread, while the detection operation of the position pointer of each caller can also be implemented through a listening thread.

[0087] In the message broker module, when a caller joins a synchronization group, if any caller in that group finishes consuming a message body and triggers its position pointer to move forward, it means that this caller will read the next message body in the message queue. Since the progress of other callers in the synchronization group reading message bodies is synchronized, in this case, in response to the event triggered by the first caller in the synchronization group to move its position pointer forward, this application not only assigns the message body pointed to by the moved position pointer to the caller that triggered the position pointer forward, but also assigns the message body to other callers in the synchronization group. This allows all caller threads in the synchronization group to read the message body and determine whether to consume it. This achieves coordination of the progress of callers reading message bodies in the same synchronization group, and also ensures that the corresponding position pointers of each caller in the synchronization group remain synchronized.

[0088] The message body in the message queue is represented in the form of binary serialized data. When the caller's position pointer moves to the next message body, the message broker module will deserialize it to obtain its corresponding object data in order to identify the data content of the message body. Then, based on this object data, it will determine whether it is compatible with the caller. Only when it is compatible will the caller consume the message body.

[0089] Since the callers in the synchronization group read the message body at the same pace, it is obviously repetitive for all these callers to deserialize the serialized data of the same message body. In this case, in one embodiment, after the first caller in the synchronization group deserializes the message body to obtain the object data, the message body can be distributed to the other callers in the synchronization group in the form of its object data, so that the other callers do not need to perform the operation of deserializing the message body, thereby saving system overhead.

[0090] Each caller can set matching rules corresponding to the message body it needs. These matching rules typically include specifying specific data objects and their corresponding eligible data characteristics, and can be pre-stored in a registration information table for on-demand retrieval. When each caller in the synchronization group receives the message body, it can filter the message body based on the object data of this message body using the caller's corresponding matching rules. Message bodies that meet the matching rules are considered eligible message bodies for consumption, and the caller consuming the eligible message body becomes an eligible caller. Therefore, when a caller needs to perform eligible matching, the message broker module performs more preprocessing processes such as deserialization and filtering of the message body. In this case, since at least the deserialization operation is reusable, each caller within the same synchronization group can reuse part or all of the preprocessing products, saving the system overhead of the broker server. In other words, the application of the synchronization group's group mechanism effectively constructs a mechanism for multiplexing the message body preprocessing process, which helps improve the efficiency of callers within the same synchronization group to quickly adapt to their corresponding message bodies, improves server operating efficiency, and thus increases the server's throughput for message call requests.

[0091] Compared to existing technologies, this application embeds a proxy mechanism between the caller and the message queue. First, the message stream from the message queue is loaded into a buffer. For each caller, based on the position pointer of the message body in the message stream, other callers with synchronized position pointers are found. All callers with synchronized position pointers are constructed into the same synchronization group. Then, whenever the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is synchronously distributed to the caller in the same synchronization group that matches that message body for consumption. Thus, by using the synchronization group mechanism to quickly match callers who need to read the message body based on whether the position pointers are synchronized, unnecessary message bodies are filtered for the caller, eliminating the hassle of individual message body preprocessing and identification by each caller in the same synchronization group. This can improve message body consumption efficiency in high-concurrency call scenarios and save unnecessary computer system overhead. For various application scenarios in e-commerce platforms that require asynchronous message processing, it can significantly improve the operating efficiency of online services provided by the caller and optimize the user experience of these online services.

[0092] Based on any embodiment of this application, please refer to Figure 5 Before determining the position pointer of the message body in the message stream within the buffer, based on the current caller's positioning reference information, the process includes:

[0093] Step S4100: Respond to the registration request of the current caller and extract the registration information carried in the registration request. The registration information includes the positioning reference information, which includes the message topic, topic partition, and message offset.

[0094] Any online service can act as the current caller and submit its registration request to the proxy server of this application, that is, the server where the message broker module of this application runs. The registration request carries the registration information corresponding to the online service.

[0095] After receiving the registration request, the message broker module of this application parses it and extracts the registration information. In this embodiment, the current caller needs to embed positioning reference information in its registration information so that the message queue it calls and its position pointer in the message queue can be determined based on this positioning reference information. Of course, in other embodiments, the current caller can also carry its corresponding matching rules in its registration information, which can be used to filter message bodies that are suitable for the current caller. Of course, the matching rules can also be submitted separately by the current caller.

[0096] In this embodiment, the instance accessed by the message broker module is determined, that is, the message server it serves is determined. Therefore, the current caller's positioning reference information can include at least three items: message topic, topic partition, and message offset. The message topic indicates the message topic used by the caller's online service, the topic partition indicates the specific storage partition belonging to the message topic that the caller needs to read, that is, the corresponding message queue, and the message offset is used to address the message body in the message stream flowing through the message queue.

[0097] Step S4200: Determine whether the source message topic and source topic partition of the message stream in the cache belongs to the message topic and topic partition specified in the positioning reference information based on the message topic and topic partition. If not, push the topic partition list to the current caller and terminate the response to the registration request.

[0098] For the message broker module of this application, it is typically responsible for interfacing with one or more instances running on a message server. Since the location baseline information does not include instance specification, it can be understood as corresponding to a default instance. In this case, the message broker module knows each topic partition in the instance it is currently interfacing with and the message topic to which it belongs. That is, the message broker module knows the source topic partition of the message stream corresponding to the one or more message queues it maintains and the source message topic to which the source topic partition belongs.

[0099] Clearly, the message stream that the proxy server where the current message broker module resides connects to must be provided by the message queue that the caller wants to access; otherwise, the caller cannot obtain the correct data. In this case, it is necessary to compare the message topic and topic partition provided by the current caller in its positioning reference information. When the message topic provided in the positioning reference information is the source message topic of the message stream corresponding to the current proxy server, and the topic partition provided in the positioning reference information is also the source topic partition of the message stream corresponding to the current proxy server, it indicates that the message stream originates from the target message queue that the current caller needs to access. Otherwise, the current proxy server does not have the target message queue that the caller needs to access.

[0100] Since the messaging service system maintains a list of topic partitions, which contains information on the deployment and distribution of each message server across the entire messaging service cluster, to ensure that the current caller can access the correct target message queue, the current message broker module can either send this list of topic partitions to the current caller so that the caller can access the correct broker server based on this list, or the current broker can query the list of topic partitions on behalf of the caller and push the corresponding broker server entry point for the target message queue to the caller. In any case, after sending the list of topic partitions to the current caller, the current broker can stop responding to the caller's registration request to save system overhead.

[0101] Step S4300: If it belongs to the message topic and topic partition specified in the positioning reference information, write the registration information of the current caller into the registration information table for future use.

[0102] Of course, if, after comparison, the message topic and topic partition specified in the current caller's location reference information happen to match the source message topic and source topic partition of a message stream in the current proxy server's cache, then it indicates that the current caller can correctly access the message body in the target message queue. Therefore, to facilitate subsequent calls, the registration information submitted by the current caller can be written into a preset registration information table for querying and calling.

[0103] As can be easily understood from the above embodiments, the pre-registration required for proxy invocation by submitting a registration request by the current caller can improve the reliability of correct message invocation. The message proxy service implemented by the message proxy module in this application can centrally control the access entry of each online service as a caller to the message queue, intervene and optimize the message invocation process, thereby ensuring that the message service system operates more reliably.

[0104] Based on any embodiment of this application, and based on the current caller's positioning reference information, the determination of the position pointer of the message body in the message stream read from the buffer by the current caller includes:

[0105] Step S5210: Initialize and determine the position pointer of the current caller in the message stream of the buffer based on the message topic, topic partition and message offset in the current caller's positioning reference information. The message body pointed to by the position pointer is the current message body.

[0106] As mentioned earlier, the corresponding message queue can be determined based on the message topic and topic partition in the current caller's positioning reference information. The message body in the topic partition can be determined based on the message offset in the positioning reference information. The encoding position of this message body is the initial position pointed to by the current caller's benefit pointer. For ease of explanation, the message body pointed to by the position pointer is called the current message body.

[0107] Step S5220: In response to the first consumption event where the current message body is consumed by the current caller, control the position pointer of the current caller to move forward relative to the current message body to the next message body in the message stream.

[0108] When the current caller finishes consuming the current message body pointed to by its position pointer, a corresponding first consumption event can be triggered. In response to this first message event, the current caller's position pointer is moved forward to the next message body. As the current caller's position pointer moves forward ahead of the position pointers of other callers in the synchronization group, the current caller can read the next message body in advance for preprocessing. When other callers in the synchronization group finish consuming the current message body, they can directly use the result of the current caller's preprocessing of the next message body, so that other callers do not have to repeat at least part of the preprocessing process of the next message body.

[0109] As can be seen from the above embodiments, for each caller, its position pointer can be dynamically maintained and updated according to the progress of its message consumption. Since each caller basically consumes the message body in the same message queue synchronously, each caller in the same synchronization group can still maintain the synchronization of their position pointers according to their fixed consumption business logic, thereby sharing some preprocessed data and saving the system overhead of the proxy server.

[0110] Based on any embodiment of this application, identifying other callers whose position pointers are synchronized with the current caller, and setting the current caller and other callers to belong to the same synchronization group, includes:

[0111] Step S5310: Determine whether the position pointer of the current caller is the same as the position pointer of any other caller. If they are the same, set the current caller and the other caller to belong to the same current synchronization group.

[0112] After the current caller initializes or consumes a message body and updates its position pointer, a corresponding pointer update event is triggered. In response to this event, it can be determined whether the position pointers of other callers accessing the same topic partition are consistent with the current caller's position pointer. If they are consistent, it means their position pointers are the same; otherwise, they are different. For any other caller whose position pointer is the same as the current caller's, they are set to belong to the same synchronization group, thus associating these callers with synchronized position pointers together. Of course, if one of the other callers already belongs to a pre-established synchronization group, the current caller can be set as a member of that group; otherwise, a new synchronization group can be created. For ease of understanding, the synchronization group to which the current caller belongs is called the current synchronization group.

[0113] Step S5320: Determine whether the position pointer of the caller in the current synchronization group is the same as the position pointer of the caller in any other synchronization group. If they are the same, merge the current synchronization group with the other synchronization group into the same synchronization group.

[0114] For multiple callers in the same message queue, there may be two or more synchronization groups with different position pointers. However, as the "convergent evolution" takes effect in the message body process of each caller, the position pointers of the callers in these different synchronization groups may gradually reach a synchronized state. At this time, in order to ensure the robustness of the group management logic, the message broker module of this application simultaneously detects the synchronization of the position pointers of each synchronization group. For example, by detecting whether the position pointer of any caller in the current synchronization group is the same as the position pointer of any caller in any other synchronization group, it can identify whether the position pointers of two different synchronization groups are actually synchronized. When it is identified that the position pointers of two synchronization groups are synchronized, the two synchronization groups can be merged into the same synchronization group, and the group identifiers of all callers in the two synchronization groups can be unified into the same synchronization group.

[0115] As can be seen from the above embodiments, this application not only coordinates the message bodies in the message queues of different callers through synchronization groups, but also detects the synchronization relationship of the position pointers of different synchronization groups in a timely manner and merges synchronization groups with synchronized position pointers in a timely manner, thereby maximizing the application efficiency of the multiplexing mechanism for message body preprocessing, effectively saving the system overhead of the proxy server, and helping to improve the throughput of the proxy server in responding to the caller's requests.

[0116] Based on any embodiment of this application, please refer to Figure 6When the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is assigned to a caller in the synchronization group that matches the message body for consumption, including:

[0117] Step S5410: After any caller in the synchronization group finishes consuming the message body at the position pointer, the position pointer is triggered to move forward to the next message body in the message stream, and the next message body is taken as the target message body;

[0118] As mentioned earlier, after a caller finishes consuming a message body, it will control its position pointer to move forward along the message flow to the next message body. Therefore, it is easy to understand that any caller in the synchronization group will trigger its corresponding position pointer to move forward to the next message body after it finishes consuming the message body pointed to by its position pointer. For the sake of explanation, in this embodiment, this next message body is referred to as the target message body.

[0119] Step S5420: Determine the caller that matches the target message body as the qualified caller according to the matching rules corresponding to each caller;

[0120] Each caller can carry its matching rules through its registration information. These rules can be used to filter message bodies that match the caller's preferences, such as message bodies where the attribute data of a data object equals a set value. In one embodiment, the target message body can be first converted into an object data format to apply the matching rules more efficiently based on the object data.

[0121] In the message broker module of this application, when a caller in a synchronization group moves its position pointer to the next message body, after performing corresponding preprocessing such as deserialization on the next message body, it can call the matching rules set by each caller in the synchronization group to determine whether the target message body is a message body that conforms to the matching rules of each caller. When the target message body is compatible with the matching rules of a certain caller, this caller can be regarded as a qualified caller of the target message body.

[0122] Step S5430: Send the target message body to each of the qualified callers for consumption.

[0123] Once the eligible callers of the target message body are identified in the synchronization group, the target message body, which can be its preprocessed data format, such as its corresponding object data, can be sent to the corresponding eligible callers for consumption. It should be noted that each caller typically consumes the message body through its own consumption thread; therefore, simply distributing the target message body to the consumption threads of each eligible caller is sufficient for consumption.

[0124] As can be seen from the above process, for each caller in the synchronization group, since the position pointers for reading the message body are synchronized among them, only one caller needs to trigger the first preprocessing of the message body pointed to by the position pointer to obtain the corresponding result data. Based on this result data, the qualified callers in the synchronization group can be quickly screened out. Finally, these result data are distributed to each qualified caller for consumption. This process merges the original repetitive operation process of each caller, significantly improves the running efficiency, and can realize the rapid matching between the message body and the caller.

[0125] Based on any embodiment of this application, determining the caller that matches the target message body as a qualified caller according to the matching rules corresponding to each caller includes:

[0126] Step S5421: Obtain the matching rules provided by each caller in the synchronization group through its registration information. The matching rules are used to specify a specific data object and its corresponding eligible data characteristics.

[0127] As described in one of the embodiments above, when a caller submits its registration request to the proxy server, it can include matching rules in the registration information carried in the registration request. The description of the matching rules is very flexible, but it usually includes the specification of a specific data object and gives the eligible data characteristics that the attribute data of the specific data object should have.

[0128] For example, a matching rule in a caller's case might be expressed as "table=my_name," where "table" refers to a specific data object, and "=my_name" represents the eligible data characteristics that the attribute data of that specified data object should match—that is, the matching condition. Similarly, a matching rule could also be in the form of "age<=100." Furthermore, multiple such matching conditions can be combined within the same matching rule, for example, "age>20 && age<=30." These and similar conditions can be flexibly configured.

[0129] Step S5422: Based on each caller as the target caller, determine whether the attribute data of a specific data object in the target message body matches the qualified data characteristics of the target caller. If they match, determine that the target caller is a qualified caller of the target message body.

[0130] In one embodiment, the producer in the message service system describes the message body using JSON, making the message body's data structure include multiple data objects and corresponding attribute data for each data object. This data is then serialized into binary data format and written to the message queue. Therefore, the message body in the message queue is typically binary data. To accommodate this, when applying the caller's matching rules, the message body can be preprocessed, for example, by deserializing its binary data into object data, and then filtering and identifying the object data based on the matching rules.

[0131] Based on the above principles, in a synchronization group, when one of the callers moves its pointer forward to the target message body, it obtains the object data of the target message body. Then, taking each caller in the synchronization group as the target caller, it determines whether the attribute data of a specific data object specified by the target caller's matching rule in the object data of the target message body meets the eligible data characteristics in the matching rule. If they match, then this target caller is identified as an eligible caller of the target message body. By analogy, all eligible callers can be determined among the callers in the synchronization group.

[0132] As can be seen from the above embodiments, when this application detects that a caller in the same synchronization group advances its position pointer to the next message body first, it uses the next message body as the target message body to obtain its object data format. By utilizing the matching rules pre-registered by each caller in the synchronization group, it matches the qualified caller based on the object data of the target message body, thereby avoiding each caller in the synchronization group from parsing the target message body separately and avoiding each caller from repeatedly parsing the target message body, ensuring that the process of selecting qualified callers is more efficient.

[0133] Based on any embodiment of this application, please refer to Figure 7 Determining whether the attribute data of a specific data object in the target message body matches the eligibility data characteristics of the target caller, and determining that the target caller is an eligible caller of the target message body when a match is found, includes:

[0134] Step S6100: Slide a sliding window of preset byte length in the serialized data corresponding to the target message body in the buffer area, and read out the serialized data corresponding to the sliding window;

[0135] Although performing a single deserialization of the same message body for different callers in the same synchronization group to obtain its object data can effectively avoid repeated preprocessing of the message body by each caller, thereby improving system efficiency, this embodiment can further improve efficiency through clever processing.

[0136] Specifically, the process of deserializing the message body and applying matching rules for filtering will be merged and optimized. To this end, a sliding window with a fixed byte length, such as 64 bytes, will be set first. Then, for the binary serialized data of the target message body that needs to determine the qualified caller, the sliding window will be applied to slide and retrieve data. The serialized data corresponding to the sliding window will be read out, and then the subsequent processing steps will be performed. For each serialized data corresponding to each slide of the sliding window, the corresponding matching rules of each caller will be applied to determine the qualified caller, until the matching of the binary data of the entire target message body is completed, or until all callers in the synchronization group are determined to be qualified callers.

[0137] Step S6200: Deserialize the serialized data corresponding to the sliding window to obtain object data, wherein the object data includes multiple data objects and their corresponding attribute data;

[0138] For the serialized data corresponding to the current position of the sliding window, deserialization is performed to convert it from binary data form to object data form, such as the object data represented in JSON language as mentioned above. In this way, multiple data objects and their corresponding attribute data carried in the object data are obtained.

[0139] Step S6300: Identify whether the object data contains a specific data object specified in the matching rule of the target caller, and whether the attribute data of the contained specific data object matches the qualified data features specified in the matching rule. When a match is found, determine that the target caller is a qualified caller of the target message body.

[0140] For the current object data obtained by deserializing the corresponding serialized data at the position of the word selected in the current sliding window, it can be compared one by one with the matching rules of each caller in the synchronization group. The specific matching principle is as described above. When the current object data contains a specific data object specified in the matching rule of a target caller, and the attribute data corresponding to this specific data object in the object data also meets the qualified data characteristics specified in the matching rule, this target caller can be determined as a qualified caller of the target message body. Therefore, in subsequent iterations of the sliding window, it is not necessary to apply matching rules to this qualified caller, and the target message body can be quickly sent to this qualified caller for consumption. In other words, by preprocessing the binary data of the target message body through segmented reading, deserialization, and filtering, the qualified caller corresponding to the target message body can be determined more quickly.

[0141] In one embodiment, adapting to the improvements of this embodiment, the message producer can place the specific data object specified by the matching rule at the beginning of the message body it produces. In this way, when determining the qualified caller, the qualified caller can usually be determined after the sliding window reads the binary data of the target message body for the first time, which is very fast and efficient.

[0142] Step S6400: When there is a mismatch, the sliding window continues to iterate and identify until the serialized data of the target message body is identified.

[0143] For any target caller in the synchronization group, if the corresponding current object data obtained by the sliding window at the current sliding position cannot match the matching rules of the target caller, then the sliding window needs to be advanced, and the process should be repeated in step S6100 until the target caller is determined to be a qualified caller, or until the sliding window reaches the end of the binary serialized data of the target message body and fails to determine the target caller to be a qualified caller, then the target caller is determined to be an ineligible caller and the target message body is not allocated to the target caller for consumption.

[0144] It is easy to understand that for all callers in the synchronization group, the above logic can be followed. When the sliding window reads a serialized data and obtains its corresponding object data each time, the corresponding matching rules of each caller are uniformly applied to filter the target message body.

[0145] As can be seen from the above embodiments, this application uses a sliding window to segment the serialized data of the target message body and convert it into object data. Based on each segment of object data, the target message body is filtered for all callers in the synchronization group. The filtering efficiency is faster and more efficient. When the specific data object limited in the matching rules is constrained to be placed at the head of the message body, the most obvious efficiency advantage can be obtained.

[0146] Please see Figure 8According to one aspect of this application, a message broker invocation apparatus includes a message loading module 5100, a pointer tracking module 5200, a group maintenance module 5300, and a matching consumption module 5400. The message loading module 5100 is configured to load a message stream from a message queue of a message server into a buffer, the message stream including multiple message bodies. The pointer tracking module 5200 is configured to determine the position pointer of the message body in the message stream read by the current caller based on the current caller's positioning reference information. The group maintenance module 5300 is configured to identify other callers whose position pointers are synchronized with the current caller and set the current caller and other callers to belong to the same synchronization group. The matching consumption module 5400, when the position pointer of any caller in the synchronization group moves forward, allocates the message body pointed to by that position pointer to a caller in the synchronization group that matches that message body for consumption.

[0147] Based on any embodiment of this application, the message broker invocation module of this application includes: a registration response module, configured to respond to the registration request of the current caller, extract the registration information carried in the registration request, the registration information including the positioning reference information, the positioning reference information including message topic, topic partition and message offset; a source identification module, configured to determine whether the source message topic and source topic partition of the message stream in the cache belongs to the message topic and topic partition specified in the positioning reference information according to the message topic and topic partition, if not, push the topic partition list to the current caller and terminate the response to the registration request; and a registration backup module, configured to write the registration information of the current caller into a registration information table for later invocation if it belongs to the message topic and topic partition specified in the positioning reference information.

[0148] Based on any embodiment of this application, the pointer tracking module 5200 includes: a pointer positioning unit, configured to initialize and determine the position pointer of the current caller in the message stream of the buffer based on the message topic, topic partition and message offset in the positioning reference information of the current caller, wherein the message body pointed to by the position pointer is the current message body; and a pointer moving unit, configured to control the position pointer of the current caller to move forward relative to the current message body to the next message body in the message stream in response to a first consumption event in which the current message body is consumed by the current caller.

[0149] Based on any embodiment of this application, the group maintenance module 5300 includes: a caller merging unit, configured to determine whether the position pointer of the current caller is the same as the position pointer of any other caller, and if so, to set the current caller and the other caller to belong to the same current synchronization group; and a group merging unit, configured to determine whether the position pointer of the caller in the current synchronization group is the same as the position pointer of the caller in any other synchronization group, and if so, to merge the current synchronization group and the other synchronization group into the same synchronization group.

[0150] Based on any embodiment of this application, the matching consumption module 5400 includes: a target determination unit, configured to trigger the position pointer to move forward to the next message body in the message stream after any caller in the synchronization group completes the consumption of the message body at the position pointer, and take the next message body as the target message body; an appropriate matching unit, configured to determine the caller that is compatible with the target message body as an appropriate caller according to the matching rules corresponding to each caller; and a message push unit, configured to send the target message body to each of the appropriate callers for consumption.

[0151] Based on any embodiment of this application, the eligible matching unit includes: a rule invocation module, configured to obtain matching rules provided by each caller in the synchronization group through its registration information, wherein the matching rules are used to specify a specific data object and its corresponding eligible data characteristics; and a rule application module, configured to determine whether the attribute data of the specific data object in the target message body matches the eligible data characteristics of the target caller based on each caller as the target caller, and when they match, determine that the target caller is an eligible caller of the target message body.

[0152] Based on any embodiment of this application, the rule application module includes: a window invocation unit, configured to slide a sliding window of preset byte length in the serialized data corresponding to the target message body in the cache area, and read the serialized data corresponding to the sliding window; a deserialization unit, configured to deserialize the serialized data corresponding to the sliding window to obtain object data, the object data including multiple data objects and their corresponding attribute data; a rule application unit, configured to identify whether the object data contains a specific data object specified in the matching rule of the target caller, and whether the attribute data of the contained specific data object matches the qualified data features specified in the matching rule, and when a match is found, determine that the target caller is a qualified caller of the target message body; and an iterative decision unit, configured to advance the sliding window to continue iterative identification when there is no match, until the identification of the serialized data of the target message body is completed.

[0153] Another embodiment of this application also provides a message broker invocation device. For example... Figure 9 The diagram shows the internal structure of a message broker invocation device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement a message broker invocation method.

[0154] The processor of the message broker invocation device provides computing and control capabilities to support the operation of the entire device. The memory of the device can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the message broker invocation method of this application. The network interface of the device is used for communication with a terminal.

[0155] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the message broker invocation device to which the solution of this application is applied. The specific message broker invocation device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0156] In this embodiment, the processor is used to execute... Figure 8 The specific functions of each module are described, and the memory stores the program code and various data required to execute the aforementioned modules or sub-modules. The network interface is used to enable data transmission between user terminals or servers. In this embodiment, the non-volatile readable storage medium stores the program code and data required to execute all modules in the message broker invocation device of this application. The server can invoke the server's program code and data to execute the functions of all modules.

[0157] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the message broker invocation method of any embodiment of this application.

[0158] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0160] In summary, this application achieves rapid matching of callers who need to read message bodies by synchronizing position pointers and leveraging a group mechanism. This can improve message body consumption efficiency, save system overhead, and significantly improve the operational efficiency of online services provided by callers.

Claims

1. A message broker invocation method, characterized in that, include: A message stream, comprising multiple message bodies, is loaded from the message queue of the message server into the buffer. Based on the current caller's positioning reference information, the position pointer of the message body in the message stream in the current caller's buffer is determined. Identify other callers whose position pointers are synchronized with the current caller, and set the current caller and other callers to belong to the same synchronization group; When the position pointer of any caller in the synchronization group moves forward, the message body pointed to by the position pointer is assigned to the caller in the synchronization group that matches the message body for consumption, and the callers in the synchronization group share the preprocessing results of the same message body.

2. The message broker invocation method according to claim 1, characterized in that, Based on the current caller's location reference information, before determining the position pointer of the message body in the message stream read by the current caller from the buffer, the process includes: In response to the current caller's registration request, the registration information carried in the registration request is extracted. The registration information includes the positioning reference information, which includes the message topic, topic partition, and message offset. Based on the message topic and topic partition, determine whether the source message topic and source topic partition of the message stream in the cache belong to the message topic and topic partition specified in the positioning reference information. If they do not belong, push the topic partition list to the current caller and terminate the response to the registration request. If the message topic and topic partition specified in the positioning reference information are included, the registration information of the current caller will be written into the registration information table for future use.

3. The message broker invocation method according to claim 2, characterized in that, Based on the current caller's location reference information, the position pointer of the message body in the message stream within the buffer is determined, including: The position pointer of the current caller in the message stream of the buffer is determined based on the message topic, topic partition and message offset in the current caller's positioning reference information. The message body pointed to by the position pointer is the current message body. In response to a first consumption event where the current message body is consumed by the current caller, the position pointer of the current caller is moved forward relative to the current message body to the next message body in the message stream.

4. The message broker invocation method according to claim 1, characterized in that, Identify other callers whose position pointers are synchronized with the current caller, and set the current caller and other callers to belong to the same synchronization group, including: Determine if the current caller's position pointer is the same as that of any other caller. If they are the same, set the current caller and that other caller to belong to the same current synchronization group. Determine whether the position pointer of the caller in the current synchronization group is the same as the position pointer of the caller in any other synchronization group. If they are the same, merge the current synchronization group with the other synchronization group into the same synchronization group.

5. The message broker invocation method according to any one of claims 1 to 4, characterized in that, When the position pointer of any caller in the synchronization group moves forward, the message body pointed to by that position pointer is assigned to the caller in the synchronization group that matches that message body for consumption, including: After any caller in the synchronization group finishes consuming the message body at the position pointer, the position pointer is triggered to move forward to the next message body in the message stream, and the next message body is taken as the target message body; The appropriate caller is determined based on the matching rules corresponding to each caller and the caller that matches the target message body. The target message body is sent to each of the eligible callers for consumption.

6. The message broker invocation method according to claim 5, characterized in that, Based on the matching rules corresponding to each caller, callers that match the target message body are identified as qualified callers, including: Obtain the matching rules provided by each caller in the synchronization group through its registration information. The matching rules are used to specify a specific data object and its corresponding eligible data characteristics. Based on each caller as the target caller, it is determined whether the attribute data of a specific data object in the target message body matches the qualified data characteristics of the target caller. When they match, the target caller is determined to be a qualified caller of the target message body.

7. The message broker invocation method according to claim 6, characterized in that, Determining whether the attribute data of a specific data object in the target message body matches the eligibility data characteristics of the target caller, and if a match is found, determining that the target caller is an eligible caller of the target message body, includes: A sliding window of preset byte length slides across the serialized data corresponding to the target message body in the buffer, and the serialized data corresponding to the sliding window is read out. The serialized data corresponding to the sliding window is deserialized to obtain object data, which includes multiple data objects and their corresponding attribute data. Identify whether the object data contains a specific data object specified in the matching rules of the target caller, and whether the attribute data of the contained specific data object matches the eligible data features specified in the matching rules. When a match is found, determine that the target caller is an eligible caller of the target message body. If a mismatch occurs, the sliding window continues to iterate and identify until the serialized data of the target message body is identified.

8. A message broker invocation device, characterized in that, include: The message loading module is configured to load a message stream from the message queue of the message server into the buffer, wherein the message stream includes multiple message bodies; The pointer tracking module is configured to determine the position pointer of the message body in the message stream within the buffer area for the current caller, based on the current caller's positioning reference information. The group maintenance module is configured to identify other callers whose position pointers are synchronized with the current caller, and set the current caller and other callers to belong to the same synchronization group; The matching consumption module, when the position pointer of any caller in the synchronization group moves forward, assigns the message body pointed to by the position pointer to the caller in the synchronization group that matches the message body for consumption, and the callers in the synchronization group share the preprocessing results of the same message body.

9. A message broker invocation device, comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images