Message queue consumption method and system, electronic device and readable medium, product

Abstract

The disclosed embodiments provide a message queue consumption method and system, an electronic device and a readable medium, and a product. The method creates a target consumption task of a target business subsystem through a queue consumption centralized management platform, and submits the target consumption task to a platform task issuing center. The platform task issuing center assigns the target consumption task to a target task execution machine in a task execution machine, obtains target consumption data of the corresponding target consumption task from a target consumption queue middleware through the target task execution machine, and calls back the target consumption data to the corresponding target business subsystem, that is, the queue consumption centralized management platform uniformly monitors the consumption queue business of the target business subsystem and calls back the target consumption data to the corresponding target business subsystem, so that the target business subsystem only needs to focus on its own business logic implementation, thereby reducing the coupling logic of each business target business subsystem and the consumption queue configuration, and reducing the complexity of development and testing.

Description

Background Art

[0002] Message queue middleware is an important component in distributed systems. It mainly solves problems such as application decoupling, asynchronous messaging, and traffic cutting, and realizes high-performance, high-availability, scalable, and eventually consistent architecture. Therefore, in system development, it is often necessary to use message queues.

[0003] However, in the related technology, on the one hand, the target business subsystem of each business separately integrates its own message queue monitoring, which makes development and testing more cumbersome; on the other hand, since the target business subsystems of each business have inconsistent mastery of the use of message queues, the code repeatability is poor, and it is inconvenient to promote usage experience.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In view of this, the present disclosure provides a message queue consumption method and system, electronic device and readable medium, and product, which can overcome the problem of cumbersome message queue development and testing of target business subsystems of various businesses due to limitations and defects of related technologies to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to the first aspect of the embodiment of the present disclosure, a message queue consumption method is proposed. The method is executed by a queue consumption centralized management platform, and the queue consumption centralized management platform includes a management background, a platform task issuing center and a task execution machine, wherein the method includes: responding to a task creation request of a target business subsystem registered with the management background through the management background, creating a target consumption task of the target business subsystem, and submitting the target consumption task to the platform task issuing center; parsing the target consumption parameters of the target consumption task through the platform task issuing center to obtain the target consumption task, determining the execution task volume of the task execution machine, and allocating the target consumption task to the target task execution machine in the task execution machine according to the execution task volume; obtaining the target consumption data corresponding to the target consumption task from the target consumption queue middleware through the target task execution machine, and calling back the target consumption data to the corresponding target business subsystem.

[0008] In an embodiment of the present disclosure, the method also includes: receiving a registration request of the target business subsystem through the management background; responding to the registration request of the target business subsystem through the management background, and registering the target consumption parameters corresponding to the target business subsystem; wherein the registration request includes at least one of the following target consumption parameters: the target system name of the target business subsystem, the target queue name of the target consumption queue corresponding to the target business subsystem in the target consumption queue middleware, the target queue communication address of the target consumption queue, the consumption callback method of the target business subsystem, the consumption callback interface address of the target business subsystem, the number of single-machine parallel threads, the single-machine consumption speed limit, and the number of consumption clients, wherein the number of single-machine parallel threads, the single-machine consumption speed limit, and the number of consumption clients are used to adjust the consumption speed of the target consumption task.

[0009] In the disclosed embodiment, the method further includes: receiving, through the management background, a parameter modification request submitted by the target business subsystem; and modifying, through the management background, the target consumption parameters registered by the target business subsystem in response to the parameter modification request.

[0010] In an embodiment of the present disclosure, the target consumption data corresponding to the target consumption task is obtained from the target consumption queue middleware through the target task execution machine, including: through the target task execution machine, according to the target queue name of the target consumption queue and the target queue communication address of the target consumption queue, a new consumption queue connection client is created to establish a connection with the target consumption queue in the target consumption queue middleware; through the target task execution machine, a new consumption task thread pool is created; and using the consumption task thread pool, the target consumption data corresponding to the target consumption task is obtained from the target consumption queue in the target consumption queue middleware.

[0011] In the embodiment of the present disclosure, the calling back of the target consumption data to the corresponding target business subsystem includes: calling back the target consumption data to the target business subsystem according to the consumption callback method of the target business subsystem and the consumption callback interface address of the target business subsystem, and through one or more of the following fields: message identifier, message body, and the international interconnection protocol IP executed this time.

[0012] In the disclosed embodiment, the method further includes: receiving receipt information fed back by the target business subsystem to indicate whether it processes the target consumption data, and the receipt information includes the following fields: status code and receipt time.

[0013] In an embodiment of the present disclosure, allocating the target consumption task to a target task execution machine among the task execution machines according to the execution task amount includes: allocating the target consumption task to a target task execution machine among the task execution machines having the least execution task amount.

[0014] According to the second aspect of the embodiment of the present disclosure, a message queue consumption method is proposed, and the method is executed by a target business subsystem, wherein the method comprises: sending a task creation request to a management background of a queue consumption centralized management platform, wherein the task creation request is used to instruct the management background to create a target consumption task for the target business subsystem, and submitting the target consumption task to a platform task issuing center of the queue consumption centralized management platform, parsing target consumption parameters of the target consumption task through the platform task issuing center, determining an execution task volume of a task execution machine of the queue consumption centralized management platform, allocating the target consumption task to a target task execution machine in the task execution machine according to the execution task volume and the target consumption parameters, obtaining target consumption data corresponding to the target consumption task from a target consumption queue middleware through the target task execution machine; and receiving the target consumption data called back by the queue consumption centralized management platform.

[0015] In the disclosed embodiment, the method further includes: the target business subsystem feeds back receipt information to the queue consumption centralized management platform to indicate whether it processes the target consumption data, and the receipt information includes the following fields: status code and receipt time.

[0016] According to the third aspect of the embodiment of the present disclosure, a queue consumption centralized management platform is proposed, including a management background, a platform task issuing center and a task execution machine, wherein the management background is used to respond to a task creation request of a target business subsystem registered in the management background, create a target consumption task of the target business subsystem, and submit the target consumption task to the platform task issuing center; the platform task issuing center is used to parse the target consumption parameters of the target consumption task, determine the execution task volume of the task execution machine, and allocate the target consumption task to the target task execution machine in the task execution machine according to the execution task volume; the task execution machine is used to obtain the target consumption data corresponding to the target consumption task from the target consumption queue middleware, and call back the target consumption data to the corresponding target business subsystem.

[0017] According to the fourth aspect of an embodiment of the present disclosure, an electronic device is proposed, which includes: one or more processors; a storage system for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the message queue consumption methods described above.

[0018] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the message queue consumption method described in any one of the above items is implemented.

[0019] According to a sixth aspect of an embodiment of the present disclosure, a computer program product is proposed, comprising: a computer program, which implements any of the message queue consumption methods described above when executed by a processor.

[0020] Certain embodiments of the present disclosure provide a message queue consumption method and system, an electronic device and a readable medium, and a product. The method creates a target consumption task of a target business subsystem through a queue consumption centralized management platform, and submits the target consumption task to a platform task issuing center. The platform task issuing center assigns the target consumption task to a target task execution machine in a task execution machine, obtains target consumption data of the corresponding target consumption task from a target consumption queue middleware through the target task execution machine, and calls back the target consumption data to the corresponding target business subsystem, that is, the queue consumption centralized management platform uniformly monitors the consumption queue business of the target business subsystem and calls back the target consumption data to the corresponding target business subsystem, so that the target business subsystem only needs to focus on its own business logic implementation, thereby reducing the coupling logic of each business target business subsystem and the consumption queue configuration, and reducing the complexity of development and testing.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. The drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work.

[0023] Figure 1 is a schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present application;

[0024] Figure 2 is a flow chart of a message queue consumption method according to an exemplary embodiment;

[0025] Figure 3 is another flow chart of a message queue consumption method according to an exemplary embodiment;

[0026] Figure 4 is another flow chart of a message queue consumption method according to an exemplary embodiment;

[0027] Figure 5 is another flow chart of a message queue consumption method according to an exemplary embodiment;

[0028] Figure 6 is a structural diagram of a queue consumption centralized management platform according to an exemplary embodiment; and

[0029] Figure 7 The present invention is a schematic diagram of the structure of an electronic device according to at least one embodiment of the present invention. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0031] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, systems, steps, etc. may be adopted. In other cases, known methods, systems, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0032] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0033] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0034] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of one or more elements / components / etc.; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0035] Figure 1 1 is a schematic diagram of a computer system provided by an exemplary embodiment of the present application. The system includes: a plurality of terminals 120 and a server cluster 140.

[0036] The terminal 120 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the terminal 120 may also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0037] The terminal 120 is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0038] The server cluster 140 may be a single server, or may include several servers, or may be a virtualization platform, or may be a cloud computing service center. The server cluster 140 is used to provide background services for the application program that provides the message queue consumption method. Optionally, the server cluster 140 undertakes the main computing work, and the terminal 120 undertakes the secondary computing work; or, the server cluster 140 undertakes the secondary computing work, and the terminal 120 undertakes the main computing work; or, the terminal 120 and the server cluster 140 adopt a distributed computing architecture for collaborative computing.

[0039] In some optional embodiments, the server cluster 140 includes a logical server 141 and a logical server 142. The logical server 141 and the logical server 142 are used to provide background services for an application program that provides a message queue consumption method.

[0040] Optionally, the client of the application installed in different terminals 120 is the same, or the client of the application installed on the two terminals 120 is the client of the same type of application of different control system platforms. Based on the different terminal platforms, the specific form of the client of the application may also be different, for example, the application client may be a mobile client, a PC client or a World Wide Web (Web) client.

[0041] Those skilled in the art will appreciate that the number of the above-mentioned terminals 120 may be more or less. For example, the above-mentioned terminal may be only one, or the above-mentioned terminals may be dozens or hundreds, or more. The embodiment of the present application does not limit the number and device type of the terminals.

[0042] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0043] Below, each step of the message queue consumption method in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0044] The embodiment of the present disclosure provides a message queue consumption method. Figure 2 The present invention is a flowchart of a message queue consumption method according to an exemplary embodiment. Figure 2 The method provided in the embodiment may be executed by a queue consumption centralized management platform, which may include a management background, a platform task distribution center, and a task execution machine.

[0045] like Figure 2 As shown, the method includes the following steps S200 to S202.

[0046] S200. In response to a task creation request of a target business subsystem registered in the management backend, the management backend creates a target consumption task for the target business subsystem and submits the target consumption task to a platform task delivery center.

[0047] S201. Analyze the target consumption parameters of the target consumption task through the platform task distribution center, determine the execution task volume of the task execution machine, and distribute the target consumption task to the target task execution machine in the task execution machine according to the execution task volume.

[0048] S202: The target consumption data corresponding to the target consumption task is obtained from the target consumption queue middleware through the target task execution machine, and the target consumption data is called back to the corresponding target business subsystem.

[0049] According to the message queue consumption method provided by the embodiment of the present disclosure, a target consumption task of a target business subsystem is created through the management background of a queue consumption centralized management platform, and the target consumption task is submitted to the platform task issuing center. The platform task issuing center assigns the target consumption task to a target task execution machine in a task execution machine, and obtains target consumption data of the corresponding target consumption task from the target consumption queue middleware through the target task execution machine, and calls back the target consumption data to the corresponding target business subsystem, that is, the queue consumption centralized management platform uniformly monitors the consumption queue of the target business subsystem, and calls back the target consumption data to the corresponding target business subsystem, so that the target business subsystem only needs to focus on its own business logic implementation, thereby reducing the coupling logic of each business target business subsystem and the consumption queue configuration, and reducing the complexity of development and testing.

[0050] The embodiment of the present disclosure provides a message queue consumption method. Figure 3 FIG. 1 is another flow chart of a message queue consumption method according to an exemplary embodiment. Figure 3 As shown, the method includes the following steps S300 to S309.

[0051] S300: Respond to the registration request of the target business subsystem through the management background, and register the target business subsystem.

[0052] The registration request of the target business subsystem is received through the management background; the registration request of the target business subsystem is responded to through the management background, and the target consumption parameters corresponding to the target business subsystem are registered. The registration request includes at least one of the following target consumption parameters: the target system name of the target business subsystem, the target queue name of the target consumption queue corresponding to the target business subsystem in the target consumption queue middleware, the target queue communication address of the target consumption queue, the consumption callback mode of the target business subsystem, the consumption callback interface address of the target business subsystem, the number of parallel threads of a single machine, the consumption speed limit of a single machine, and the number of consumption clients.

[0053] The target queue name of the target consumer queue corresponding to the target business subsystem in the target consumer queue middleware and the target queue communication address of the target consumer queue are used by the target task execution machine to determine the target consumer queue in the target consumer queue middleware.

[0054] The target system name of the target business subsystem, the consumption callback method of the target business subsystem, and the consumption callback interface address of the target business subsystem are used by the target task execution machine to determine the target business subsystem.

[0055] The number of parallel threads on a single machine, the consumption speed limit on a single machine, and the number of consumer clients are used to adjust the consumption speed of the target consumption task. For example, the number of parallel threads on a single machine refers to the number of threads on a single machine in a parallel state, the consumption speed limit on a single machine refers to the maximum consumption value of a single machine, and the number of consumer clients refers to the number of machines used to consume the task. In other embodiments of the present disclosure, other parameters may also be registered, which are not limited here.

[0056] For example, the target business subsystem sends a registration request to the management background of the queue consumption centralized management platform for registration. The registered target consumption parameters may include but are not limited to the target system name (project) of the target business subsystem, the target queue name (queenName) of the target consumption queue, the target queue communication address (mqConfig) of the target consumption queue, the number of exception retries (times), the consumption callback method of the target business subsystem (rpc (remote produce call, remote procedure call) / http), the consumption callback interface address (invokeUrl) of the target business subsystem, the consumption offset (offset setting), the timeout alarm callback address (timeOutUrl), the number of parallel threads of a single machine (threadCount), the single-machine consumption speed limit (vLimit), the number of consumption clients (consumerCount), exception strategies (such as suspending consumption, ignoring consumption, single-threaded consumption), etc.

[0057] For example, the above parameters can be configured one by one, for example, the number of consumer threads, consumer queue information, consumer callback interface address, single-machine consumer frequency, timeout alarm callback address, etc. are configured in sequence. The number of consumer threads is the setting of the number of consumer threads contained in the consumer task thread pool, and the single-machine consumer frequency refers to the consumer rate limited by a single machine when executing a consumer task. Both are used to control the consumption speed / consumption rate of the consumer task.

[0058] S301. The management backend verifies the registered target consumption parameters.

[0059] The management backend of the queue consumption centralized management platform verifies the registered target consumption parameters. For example, the management backend verifies the registered target business subsystem's consumption callback interface address. If the management backend can connect to the target business subsystem through the registered target business subsystem's consumption callback interface address, the verification passes. Other parameters may also be verified in other embodiments of the present disclosure, which are not limited here.

[0060] For example, if the queue consumption centralized management platform receives the receipt information fed back by the corresponding target business subsystem, it means that the registered consumer callback interface address (invokeUrl) has been verified, that is, the registration is successful. If the queue consumption centralized management platform does not receive the receipt information fed back by the corresponding target business subsystem, it means that the registered consumer callback interface address (invokeUrl) has not been verified, that is, the registration has failed. The relationship between the target business subsystem and the consumption management platform is established through the configuration registration of the target business subsystem in the management background.

[0061] The method of the embodiment of the present disclosure also includes modifying the target consumption parameters registered by the target business subsystem. For example, when it is necessary to adjust the registered parameters such as the consumption parameters, consumption speed, and consumption order of the target consumption queue, the registered target consumption parameters can be modified directly. For example, the parameter modification request submitted by the user through the target business subsystem is received through the management background, and the target consumption parameters registered by the target business subsystem are modified in response to the parameter modification request through the management background. The target business subsystem of the business in the related technology is non-dynamically adjusted for the configuration of message queue consumption. In its conventional development, many parameters are fixed after being written. If they need to be modified, the code needs to be modified and the service needs to be redeployed, that is, each modification requires restarting the service. In contrast, the target business subsystem of the embodiment of the present disclosure can dynamically modify the target consumption parameters of its own target consumption queue to adjust the consumption speed, consumption order, etc., and the queue consumption centralized management platform is used to realize the dynamic adjustment of consumption parameters.

[0062] S302: The management backend creates a target consumption task for the target business subsystem in response to the task creation request of the target business subsystem.

[0063] When the parameters such as the consumption callback interface address of the target business subsystem are verified, the management background creates a consumption task based on the registration of the target business subsystem.

[0064] For example, create a new consumption task, modify consumption task, or delete consumption task for each target business subsystem. For example, create a new consumption task for target business subsystem A to increase the consumption of target consumption data by target business subsystem A, create a modify consumption task for target business subsystem B to modify the consumption of target consumption data by target business subsystem B, and create a delete consumption task for target business subsystem C to delete the consumption of target consumption data by target business subsystem C.

[0065] S303: The management backend submits the target consumption task to the platform task distribution center.

[0066] S304. Analyze the target consumption parameters of the target consumption task through the platform task distribution center.

[0067] The target consumption parameters of the target consumption task are analyzed by the platform task distribution center so as to consume the target consumption task according to the target consumption parameters.

[0068] S305. The platform task distribution center determines the execution task volume of the task execution machine, and distributes the target consumption task to the target task execution machine with the least execution task volume among the task execution machines.

[0069] The platform task distribution center allocates consumption tasks according to the target consumption task volume configured by the target consumption task. For example, when it is determined that the target consumption task volume is large and the target task execution machine is insufficient to execute it, the target task execution machine can be expanded, that is, the target task execution machine can be increased.

[0070] The platform task distribution center analyzes the received new target consumption tasks, and distributes the target consumption tasks to the target task execution machine with the least amount of tasks among the task execution machines according to the amount of tasks executed by the target task execution machine. The target task execution machine with the least amount of tasks executed in the present disclosure may be a target task execution machine with a small amount of tasks executed. For example, one of multiple target task execution machines with a small amount of tasks executed is determined as the target task execution machine, and the target consumption tasks are distributed to the target task execution machine with a small amount of tasks executed.

[0071] S306. The target task execution machine creates a new consumption queue to connect to the client and a consumption task thread pool according to the target consumption task.

[0072] A new consumer queue connection client is created through the target task execution machine according to the target queue name of the target consumer queue and the target queue communication address of the target consumer queue, so as to establish a connection with the target consumer queue in the target consumer queue middleware, and obtain the target consumption data of the corresponding target consumer task from the target consumer queue middleware. At the same time, a new consumer task thread pool is created through the target task execution machine, and the consumer task thread pool is used to obtain the target consumption data of the corresponding target consumer task from the target consumer queue in the target consumer queue middleware.

[0073] S307, the target task execution machine connects to the client through the consumption queue to obtain the target consumption data of the corresponding consumption task from the target consumption queue middleware, and places it in the task thread pool.

[0074] The target task execution machine connects to the client through the consumption queue and uniformly monitors the target consumption queues corresponding to all registered target business subsystems in the target consumption queue middleware, determines the target consumption queue corresponding to the target consumption task, obtains the target consumption data of the corresponding consumption task from the target consumption queue, and places it in the task thread pool.

[0075] S308. The target task execution machine calls back the target consumption data in the task thread pool to the corresponding target business subsystem.

[0076] The target task execution machine calls back the target consumption data in the task thread pool to the target business subsystem corresponding to the target consumption task. For example, according to the consumption callback method and the consumption callback interface address of the target business subsystem, the target consumption data is called back to the corresponding target business subsystem through one or more of the following fields: message identifier, message body, and this execution of the Internet Protocol (IP). For example, the management platform mainly sends the message to the interface of the target business subsystem through the following fields: msgId (message identifier, main callback idempotent operation), msgContet (message body), and invokeIp (this execution IP).

[0077] S309: The target task execution machine receives the receipt information fed back by the corresponding target business subsystem to indicate whether it processes the target consumption data.

[0078] The target task execution machine receives the receipt information fed back by the corresponding target business subsystem to indicate whether it processes the target consumption data. For example, the target task execution machine cyclically pulls the receipt information fed back by the target business subsystem. The target task execution machine feeds back the receipt information to the platform task issuing center, and the platform task issuing center further feeds back the receipt information to the management background. The receipt information includes one or more of the following fields: status code and receipt time. The status code includes a normal status code to indicate that the target consumption data has been processed normally, and an abnormal status code to indicate that the target consumption data has been processed abnormally. For example, when the status code fed back by the corresponding target business subsystem is status code 200 and receipt time, it indicates that the target consumption data has been processed normally; when the status code fed back by the corresponding target business subsystem is status code 300, abnormal reason and receipt time, it indicates that the target consumption data has been processed abnormally.

[0079] It should be noted that the message queue middleware in the embodiments of the present disclosure includes ActiveMQ, RabbitMQ, ZeroMQ, Kafka, MetaMQ, RocketMQ, etc., but is not limited to these.

[0080] The message queue consumption method disclosed in the present invention is described in detail below with reference to specific examples.

[0081] Figure 4 FIG. 1 is another flow chart of a message queue consumption method according to an exemplary embodiment. Figure 4 As shown, the method includes the following steps S400 to S409.

[0082] S400. Subsystem A sends a registration request to the management background to register.

[0083] The management backend responds to the registration request of subsystem A and registers subsystem A.

[0084] S401, the management backend configures the number of consumption threads.

[0085] S402, the management backend configures the consumption queue related information.

[0086] S403: The management backend configures the consumption callback interface address.

[0087] S404. Configure the single-machine consumption frequency in the management backend.

[0088] S405: The management backend configures the timeout alarm callback address.

[0089] The parameters of subsystem A can be configured one by one through the order of steps S401 to S405, for example, the number of consumption threads, consumption queue information, consumption callback interface address, single-machine consumption frequency, and timeout alarm callback address are configured in sequence. In other embodiments of the present disclosure, the registered parameters can also be configured in other orders, which are not limited here.

[0090] S406: The management backend submits the target consumption task to the platform task distribution center.

[0091] S407: The platform task distribution center analyzes the target consumption parameters and allocates the target task execution machine.

[0092] S408. The platform task delivery center delivers the target consumption task to the designated target task execution machine.

[0093] S409: The target task execution machine creates a new consumption queue to connect to the client according to the target consumption task.

[0094] S410: The target task execution machine creates a new consumer task thread pool according to the target consumer task.

[0095] S411. The target task execution machine obtains target consumption data corresponding to the consumption task from the target consumption queue middleware.

[0096] S412: The target task execution machine calls back the target consumption data in the task thread pool to the corresponding target business subsystem.

[0097] S413: The target task execution machine receives receipt information fed back by the corresponding target business subsystem.

[0098] It should be noted that the relevant technical solutions of the above steps S400 to S413 of the embodiment of the present disclosure can refer to the above Figure 3 The corresponding steps of the illustrated embodiment have similar technical principles and technical effects, which will not be repeated here. In addition, the implementation steps of the message queue consumption method of the embodiment of the present disclosure are not limited to the above steps. In other embodiments of the present disclosure, other steps can also be used to implement the message queue consumption method of the present disclosure, which is not limited here.

[0099] According to the message queue consumption method provided by the embodiment of the present disclosure, a target consumption task of a target business subsystem is created through a queue consumption centralized management platform, and the target consumption task is submitted to a platform task issuing center. The platform task issuing center assigns the target consumption task to a target task execution machine in a task execution machine, and obtains target consumption data of the corresponding target consumption task from the target consumption queue middleware through the target task execution machine, and calls back the target consumption data to the corresponding target business subsystem, that is, the queue consumption centralized management platform uniformly monitors the consumption queue business of the target business subsystem and calls back the target consumption data to the corresponding target business subsystem. At the same time, the target business subsystem can dynamically modify the consumption parameters of its own consumption queue, adjust the consumption speed, consumption order, etc., and the queue consumption centralized management platform is used to realize the dynamic adjustment of consumption parameters, so that each target business subsystem only needs to focus on its own business logic implementation, thereby reducing the coupling logic of each business target business subsystem and the consumption queue configuration, and reducing the complexity of development and testing.

[0100] The embodiment of the present disclosure provides a message queue consumption method. Figure 5 FIG. 1 is a flow chart of a message queue consumption method according to an exemplary embodiment. Figure 5 As shown, the method includes the following steps S500 to S501.

[0101] S500. The target business subsystem is registered in the management background.

[0102] For example, the target business subsystem is registered in the management background of the queue consumption centralized management platform. The target business subsystem sends a task creation request to the management background of the queue consumption centralized management platform. The task creation request is used to instruct the management background to create a target consumption task for the target business subsystem, and submit the target consumption task to the platform task distribution center of the queue consumption centralized management platform. The platform task distribution center parses the target consumption parameters of the target consumption task, determines the execution task volume of the task execution machine of the queue consumption centralized management platform, and assigns the target consumption task to the target task execution machine in the task execution machine according to the execution task volume and the target consumption parameters. The target consumption data of the corresponding target consumption task is obtained from the target consumption queue middleware through the target task execution machine.

[0103] S501. The target business subsystem receives target consumption data called back by the queue consumption centralized management platform.

[0104] For example, the target business subsystem feeds back receipt information to the queue consumption centralized management platform to indicate whether it processes the target consumption data. The receipt information includes the following fields: status code and receipt time.

[0105] It should be noted that Figure 5The message queue consumption method executed by the target business subsystem shown in Figures 2 to 4 The methods on the opposite side of the message queue consumption method executed by the queue consumption centralized management platform shown can refer to each other and are not limited here.

[0106] According to the message queue consumption method provided by the embodiment of the present disclosure, the target business subsystem is registered in the management background, and receives the target consumption data called back by the queue consumption centralized management platform, that is, the queue consumption centralized management platform uniformly monitors the consumption queue business of the target business subsystem and calls back the target consumption data to the corresponding target business subsystem, so that the target business subsystem only needs to focus on its own business logic implementation, thereby reducing the coupling logic of each business target business subsystem and the consumption queue configuration, and reducing the development complexity.

[0107] The disclosed embodiment provides a queue consumption centralized management platform. Figure 6 FIG. 1 is a schematic diagram of the structure of a queue consumption centralized management platform according to at least one embodiment of the present disclosure. Figure 6 As shown, the queue consumption centralized management platform includes a management backend 601, a platform task distribution center 602, and a task execution machine 603. The management backend 601 is used to respond to the task creation request of the target business subsystem registered in the management backend, create a target consumption task of the target business subsystem, and submit the target consumption task to the platform task distribution center; the platform task distribution center 602 is used to parse the target consumption parameters of the target consumption task, determine the execution task volume of the task execution machine, and assign the target consumption task to the target task execution machine in the task execution machine according to the execution task volume; the task execution machine 603 is used to obtain the target consumption data of the corresponding target consumption task from the target consumption queue middleware, and call back the target consumption data to the corresponding target business subsystem.

[0108] The management platform 601 of the queue consumption centralized management platform includes multiple administrators (admin), the platform task issuing center 602 includes multiple task issuing subjects (master), and the task execution machine 603 includes multiple target task execution machines (worker). For example, the administrator includes the first administrator 604 and the second administrator 605, the task issuing subject includes the first task issuing subject 606 and the second task issuing subject 607, and the target task execution machine includes the first target task execution machine 608, the second target task execution machine 609 and the third target task execution machine 610. The administrator admin is mainly for users, and users can register the target business subsystem and the target consumption queue in the administrator admin background. For example, users can register the target system name of the target business subsystem of each target business subsystem and the target consumption queue, the target queue name of the target consumption queue corresponding to the target business subsystem in the target consumption queue middleware, the target queue communication address of the target consumption queue, the consumption callback method of the target business subsystem, the consumption callback interface address of the target business subsystem, the number of parallel threads of a single machine, the single machine consumption speed limit, and the number of consumption clients in the administrator admin background. After the registration is completed, the management backend creates consumption tasks according to the registration of the target business subsystem. For example, it creates new consumption tasks, modifies consumption tasks, or deletes consumption tasks for each target business subsystem.

[0109] The administrator admin submits the consumption task to the task issuing entity master. For example, the first administrator 604 submits the consumption task to the first task issuing entity 606 and the second task issuing entity 607 of the management platform, or only submits the consumption task to the first task issuing entity 606.

[0110] The task issuing subject master is used to receive the target consumption tasks delivered by the administrator admin. The task issuing subject maser will parse the target consumption parameters of the received new target consumption tasks, determine the execution task volume of the task execution machine, and assign the target consumption tasks to the target task execution machine worker with less execution task volume among the task execution machines. For example, Figure 6 As shown, the task issuing entity maser determines that the first target task execution machine worker 608 and the third target task execution machine worker 610 among multiple task execution machine workers are target task execution machine workers with fewer tasks, and it issues the target consumption task to the first target task execution machine worker 608 and the third target task execution machine worker 610, but does not issue the consumption task to the second target task execution machine worker 609 with more tasks.

[0111] The target task execution machine worker performs the actual queue consumption scheduling service. The target task execution machine worker creates a new consumption queue to connect the client (client) and the consumption task thread pool according to the target consumption task, and initializes the configured target consumption queue (Message Queue, mq). For example, the target task execution machine worker establishes a communication connection with mq, that is, establishes a communication session. The target task execution machine worker parses the target consumption parameters of the target consumption task, and obtains the target consumption data of the corresponding target consumption task from the target consumption queue middleware through the consumption queue connection client, and places it in the consumption task thread pool, and further calls back the target consumption data to the corresponding target business subsystem. For example, Figure 6 As shown, when the target business subsystem is the target business subsystem A, the first target task execution machine worker 608 and the third target task execution machine worker 610 obtain the target consumption data of the corresponding consumption task from the target consumption queue middleware through the consumption queue connection client, and place it in the consumption task thread pool, and then call back the target consumption data from the consumption task thread pool to the corresponding target business subsystem A; when the target business subsystem is the target business subsystem B, the first target task execution machine worker 608 and the third target task execution machine worker 610 obtain the target consumption data of the corresponding consumption task from the target consumption queue middleware through the consumption queue connection client, and place it in the consumption task thread pool, and then call back the target consumption data from the consumption task thread pool to the corresponding target business subsystem B.

[0112] When the user closes the target consumption task, the thread pool corresponding to the target consumption task is closed. When the user expands the capacity of the target task execution machine, the management platform sends the consumption task to the worker of the target task execution machine. When the user adjusts the relevant parameters of the thread pool, the corresponding worker of the target task execution machine assigned to the target consumption task dynamically adjusts the relevant configuration of the thread pool accordingly.

[0113] It should be noted that in the relevant technology, each business subsystem needs to introduce the relevant dependency packages of the connection consumption queue middleware mq in the program, then write the connection queue configuration, and then execute the consumption, and adjust the consumption speed of the consumption queue middleware mq by itself. After the consumption management platform is established, the business subsystem does not need to connect to the message queue by itself, nor does it need to adjust the consumption speed by itself, but it is handled by the queue consumption centralized management platform, which consumes the target consumption queue of the corresponding target consumption queue middleware mq, and after obtaining the target consumption data, the target consumption data is called back to the target business subsystem through the registered interface.

[0114] According to the queue consumption centralized management platform provided by the embodiment of the present disclosure, a target consumption task of a target business subsystem is created through the queue consumption centralized management platform, and the target consumption task is submitted to the platform task issuing center. The platform task issuing center assigns the target consumption task to a target task execution machine in a task execution machine, obtains target consumption data of the corresponding target consumption task from the target consumption queue middleware through the target task execution machine, and calls back the target consumption data to the corresponding target business subsystem, that is, the queue consumption centralized management platform uniformly monitors the consumption queue business of the target business subsystem and calls back the target consumption data to the corresponding target business subsystem. At the same time, the target business subsystem can dynamically modify the consumption parameters of its own consumption queue, adjust the consumption speed, consumption order, etc., and the queue consumption centralized management platform is used to realize the dynamic adjustment of consumption parameters, so that each target business subsystem only needs to focus on its own business logic implementation, thereby reducing the coupling logic of each business target business subsystem and the consumption queue configuration, and reducing the development complexity.

[0115] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0116] Figure 7 This is a schematic diagram of the structure of an electronic device according to at least one embodiment of the present disclosure. Figure 7 An electronic device 700 according to this embodiment of the present invention will be described. Figure 7 The electronic device 700 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0117] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0118] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps of various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can perform the following steps: Figure 2As shown in S200, create a consumption task; S201, obtain the target consumption data corresponding to the consumption task from the target consumption queue middleware, and call back the target consumption data to the corresponding target business subsystem.

[0119] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0120] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0121] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0122] The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0123] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0124] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0125] A program product for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0126] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0127] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0128] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0129] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0130] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0131] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0132] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0133] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A message queue consumption method, characterized in that: The method is executed by a queue consumption centralized management platform, which includes a management background, a platform task distribution center, and a task execution machine, wherein the method includes: In response to the task creation request of the target business subsystem, the management backend creates a target consumption task for the target business subsystem, and submits the target consumption task to the platform task issuing center; The target consumption parameters of the target consumption task are analyzed by the platform task issuing center to determine the execution task amount of the task execution machine, and the target consumption task is allocated to the target task execution machine in the task execution machine according to the execution task amount, wherein the target consumption parameters include the target queue name of the target consumption queue corresponding to the target business subsystem in the target consumption queue middleware, and the target queue communication address of the target consumption queue; The target consumption data corresponding to the target consumption task is obtained from the target consumption queue middleware through the target task execution machine, which includes: through the target task execution machine, according to the target queue name of the target consumption queue and the target queue communication address of the target consumption queue, a new consumption queue connection client is created to establish a connection with the target consumption queue in the target consumption queue middleware; through the target task execution machine, a new consumption task thread pool is created; using the consumption task thread pool, the target consumption data corresponding to the target consumption task is obtained from the target consumption queue in the target consumption queue middleware; and the target consumption data is called back to the corresponding target business subsystem.

2. The method according to claim 1, characterized in that: The method further comprises: Receiving a registration request of the target business subsystem through the management backend; Responding to the registration request of the target business subsystem through the management background to register the target consumption parameter corresponding to the target business subsystem; The registration request includes at least one of the following target consumption parameters: the target system name of the target business subsystem, the target queue name of the target consumption queue corresponding to the target business subsystem in the target consumption queue middleware, the target queue communication address of the target consumption queue, the consumption callback mode of the target business subsystem, the consumption callback interface address of the target business subsystem, the number of parallel threads of a single machine, the consumption speed limit of a single machine, and the number of consumption clients; Among them, the number of parallel threads of a single machine, the consumption speed limit of a single machine, and the number of consumption clients are used to adjust the consumption speed of the target consumption task.

3. The method according to claim 2, characterized in that: The method further comprises: Receiving, through the management backend, a parameter modification request submitted by the target business subsystem; The target consumption parameter registered by the target business subsystem is modified by the management background in response to the parameter modification request.

4. The method according to claim 2, characterized in that: The calling back the target consumption data to the corresponding target business subsystem includes: According to the consumption callback mode of the target business subsystem and the consumption callback interface address of the target business subsystem, the target consumption data is called back to the target business subsystem through one or more of the following fields: message identifier, message body, and the Internet protocol IP for this execution.

5. The method according to claim 1, characterized in that: The method further comprises: Receive the receipt information fed back by the target business subsystem to indicate whether it processes the target consumption data, The receipt information includes the following fields: status code and receipt time.

6. The method according to claim 1, characterized in that: Allocating the target consumption task to a target task execution machine among the task execution machines according to the execution task amount includes: The target consumption task is allocated to a target task execution machine that executes the least amount of tasks among the task execution machines.

7. A message queue consumption method, characterized in that: The method is performed by the target service subsystem, wherein the method comprises: A task creation request is sent to the management background of the queue consumption centralized management platform, wherein the task creation request is used to instruct the management background to create a target consumption task for the target business subsystem, and submit the target consumption task to the platform task issuing center of the queue consumption centralized management platform, and the target consumption parameter of the target consumption task is parsed by the platform task issuing center to determine the execution task amount of the task execution machine of the queue consumption centralized management platform, and the target consumption task is allocated to the target task execution machine in the task execution machine according to the execution task amount and the target consumption parameter, wherein the target consumption parameter includes the target consumption parameter corresponding to the target business subsystem in the target consumption queue middleware. The target queue name of the target consumption queue and the target queue communication address of the target consumption queue; the target consumption data corresponding to the target consumption task is obtained from the target consumption queue middleware through the target task execution machine, which includes: through the target task execution machine, according to the target queue name of the target consumption queue and the target queue communication address of the target consumption queue, a new consumption queue connection client is created to establish a connection with the target consumption queue in the target consumption queue middleware; through the target task execution machine, a new consumption task thread pool is created; using the consumption task thread pool, the target consumption data corresponding to the target consumption task is obtained from the target consumption queue in the target consumption queue middleware; Receive the target consumption data called back by the queue consumption centralized management platform.

8. The method according to claim 7, characterized in that: The method further comprises: Feedback receipt information to the queue consumption centralized management platform to indicate whether it processes the target consumption data, The receipt information includes the following fields: status code and receipt time.

9. A queue consumption centralized management platform, characterized in that: Including management background, platform task distribution center and task execution machine, The management backend is used to respond to a task creation request of a target business subsystem registered in the management backend, create a target consumption task for the target business subsystem, and submit the target consumption task to a platform task issuing center; The platform task issuing center is used to parse the target consumption parameters of the target consumption task, determine the execution task volume of the task execution machine, and distribute the target consumption task to the target task execution machine in the task execution machine according to the execution task volume, wherein the target consumption parameters include the target queue name of the target consumption queue corresponding to the target business subsystem in the target consumption queue middleware, and the target queue communication address of the target consumption queue; The task execution machine is used to obtain the target consumption data corresponding to the target consumption task from the target consumption queue middleware, which includes: through the target task execution machine, according to the target queue name of the target consumption queue and the target queue communication address of the target consumption queue, a new consumption queue connection client is created to establish a connection with the target consumption queue in the target consumption queue middleware; through the target task execution machine, a new consumption task thread pool is created; using the consumption task thread pool, the target consumption data corresponding to the target consumption task is obtained from the target consumption queue in the target consumption queue middleware; and the target consumption data is called back to the corresponding target business subsystem.

10. An electronic device, characterized in that: include: one or more processors; a storage system for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6 or 7-8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 or 7 to 8 is implemented.

12. A computer program product, characterized in that include: A computer program, characterized in that when the computer program is executed by a processor, it implements the method as described in any one of claims 1-6 or 7-8.

Images