Method, device, medium and electronic device for dynamically adjusting data consumption thread

Abstract

The present application discloses a method, device, computer-readable storage medium and electronic device for dynamically adjusting data consumption threads, which obtain current configuration information representing data consumption needs, and then compare the current configuration information with the previous configuration information to obtain change parameters, and finally adjust the data consumption thread group corresponding to the change parameters in real time according to the change parameters; that is, by obtaining the current configuration information in real time and comparing it with the previous configuration information to obtain the change parameters, and then adjusting the data consumption thread group corresponding to the change parameters in real time according to the change parameters, thereby realizing dynamic adjustment of consumption data on the data consumption system side, and improving the efficiency of data consumption thread adjustment and customer experience.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, computer-readable storage medium and electronic device for dynamically adjusting a data consumption thread. Background Art

[0002] With the continuous development of automated industrial technology, more and more automated industrial equipment has been put into use. In order to ensure the complete use of automated industrial equipment, multiple sensors are usually installed on the automated industrial equipment to collect operation data.

[0003] In order to ensure the integrity of the data, all the data of automated industrial equipment is often collected. However, in actual use, due to different needs, only part of the data needs to be selected, that is, a data consumption thread or thread group is established to achieve the effect of consuming (obtaining) data. Currently, the commonly used data stream technologies include Kafka data stream technology. However, Kafka data stream cannot dynamically control the start or stop of the consumption thread according to the changing parameters, resulting in the data consumption system (such as the client) cannot dynamically adjust the content of the consumption data. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a method, device, computer-readable storage medium and electronic device for dynamically adjusting data consumption threads, which solves the problem that the above data consumption system cannot dynamically adjust the consumption data content.

[0005] According to one aspect of the present application, a method for dynamically adjusting a data consumption thread is provided, comprising: obtaining current configuration information; wherein the configuration information represents data consumption demand; comparing the current configuration information with the previous configuration information to obtain a change parameter; wherein the change parameter is a parameter of the current configuration information that has changed relative to the previous configuration information; and adjusting, in real time according to the change parameter, a data consumption thread group corresponding to the change parameter.

[0006] In one embodiment, the obtaining of the current configuration information includes: obtaining the current configuration information when a broadcast event representing adjustment of the configuration information is received.

[0007] In one embodiment, the comparing the current configuration information with the previous configuration information includes: extracting the subject configuration information in the current configuration information and the subject configuration information in the previous configuration information; and comparing the subject configuration information in the current configuration information and the subject configuration information in the previous configuration information; wherein, adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter includes: when the change parameter includes a first subject that exists in both the previous configuration information and the current configuration information, and there is a change between the first subject configuration information in the current configuration information and the first subject configuration information in the previous configuration information, restarting the consumption thread group corresponding to the first subject.

[0008] In one embodiment, restarting the consumer thread group corresponding to the first topic includes: adjusting the first topic configuration information according to the current configuration information, and restarting the consumer thread group corresponding to the first topic.

[0009] In one embodiment, the first topic configuration information includes partition information; wherein, adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter includes: when there is a change between the data consumption thread group corresponding to the first topic in the previous configuration information and the data consumption thread group corresponding to the first topic in the current configuration information, adjusting the partition information of the first topic configuration information in the current configuration information.

[0010] In one embodiment, the real-time adjustment of the data consumption thread group corresponding to the change parameter according to the change parameter includes: when the change parameter includes second topic configuration information that exists in the previous configuration information and does not exist in the current configuration information, stopping the consumption thread group corresponding to the second topic configuration information.

[0011] In one embodiment, the real-time adjustment of the data consumption thread group corresponding to the change parameter according to the change parameter includes: when the change parameter includes third topic configuration information that does not exist in the previous configuration information and exists in the current configuration information, starting the consumption thread group corresponding to the third topic configuration information.

[0012] According to another aspect of the present application, a dynamic adjustment device for data consumption threads is provided, including: an acquisition module, used to acquire current configuration information; wherein the configuration information represents data consumption demand; a comparison module, used to compare the current configuration information with the previous configuration information to obtain a change parameter; wherein the change parameter is a parameter of the current configuration information that has changed relative to the previous configuration information; and an adjustment module, used to adjust the data consumption thread group corresponding to the change parameter in real time according to the change parameter.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute any of the above-mentioned methods for dynamically adjusting data consumption threads.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and the processor for executing any of the above-mentioned methods for dynamically adjusting data consumption threads.

[0015] The present application provides a method, device, computer-readable storage medium and electronic device for dynamically adjusting data consumption threads, which obtain current configuration information representing data consumption needs, then compare the current configuration information with the previous configuration information to obtain change parameters, and finally adjust the data consumption thread group corresponding to the change parameters in real time according to the change parameters; that is, by obtaining the current configuration information in real time and comparing it with the previous configuration information to obtain the change parameters, and then adjusting the data consumption thread group corresponding to the change parameters in real time according to the change parameters, thereby realizing dynamic adjustment of consumption data on the data consumption system side, and improving the efficiency of data consumption thread adjustment and customer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 It is a flowchart of a method for dynamically adjusting a data consumption thread provided by an exemplary embodiment of the present application.

[0018] Figure 2 It is a flowchart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application.

[0019] Figure 3 It is a flowchart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application.

[0020] Figure 4 It is a flowchart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application.

[0021] Figure 5It is a flowchart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application.

[0022] Figure 6 It is a structural diagram of a device for dynamically adjusting a data consumption thread provided by an exemplary embodiment of the present application.

[0023] Figure 7 It is a structural diagram of a dynamic adjustment device for data consumption threads provided by another exemplary embodiment of the present application.

[0024] Figure 8 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0025] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0026] Application Overview

[0027] Streaming technology applications consume queue data and perform real-time calculations in memory, and finally send out the calculated data information and processed information such as alarms. Kafka Streaming is the most commonly used framework for stream processing and analysis, but the current standard Kafka Streaming has the following two problems:

[0028] 1. Unable to autonomously control the start and stop of consumer threads or thread groups.

[0029] 2. It is not possible to dynamically create consumer thread groups for related topics based on configuration.

[0030] In order to enable Kafka Streaming to respond to configuration information changes in a timely manner, this application configures Kafka consumer topics and consumer thread group information by adding a new configuration database and configuring parameters through an interface; and when the configuration changes, the corresponding Kafka consumer group program adjusts the consumer group threads to achieve the effect of dynamically adjusting consumer threads.

[0031] The following describes in detail the implementation of the method, device, computer-readable storage medium and electronic device for dynamically adjusting data consumption threads provided by the present application in conjunction with the accompanying drawings.

[0032] Exemplary Methods

[0033] Figure 1FIG. 1 is a flow chart of a method for dynamically adjusting a data consumption thread provided by an exemplary embodiment of the present application. Figure 1 As shown, the dynamic adjustment method of the data consumption thread includes:

[0034] Step 110: Obtain current configuration information; wherein the configuration information represents data consumption requirements.

[0035] Since the same industrial equipment data may be used by one or more users, different data streams (corresponding to different threads or thread groups) are usually used for different users to avoid mutual interference between multiple users. Since users may be added or deleted from the system, that is, new data streams may be started or deleted, or users may modify their configurations (such as the content or type of data consumption) according to their needs, at this time, it is necessary to synchronously obtain the current configuration information to obtain the current data stream demand information in real time.

[0036] In one embodiment, the specific implementation of step 110 may be: when a broadcast event representing the adjustment of configuration information is received, the current configuration information is obtained. When the configuration information changes, the corresponding consumer thread group program is notified to adjust the consumer thread by publishing an event (for example, using a message middleware to broadcast an event of configuration information change). When the broadcast event is received, the current configuration information is immediately obtained.

[0037] Step 120: Compare the current configuration information with the previous configuration information to obtain the changed parameters; wherein the changed parameters are the parameters of the current configuration information that have changed relative to the previous configuration information.

[0038] After the current configuration information is obtained, the current configuration information is compared with the previous configuration information (cached or stored in the system) to obtain the difference between the current configuration information and the previous configuration information, that is, to obtain the changed parameters.

[0039] Step 130: According to the changed parameters, the data consumption thread group corresponding to the changed parameters is adjusted in real time.

[0040] The data consumption thread group that needs to be adjusted is determined based on the change parameter, and the corresponding data consumption thread group is dynamically adjusted according to the change parameter to meet the current configuration information requirement.

[0041] The present application provides a method for dynamically adjusting data consumption threads, which obtains current configuration information that represents data consumption needs, compares the current configuration information with the previous configuration information, obtains change parameters, and finally adjusts the data consumption thread group corresponding to the change parameters in real time according to the change parameters; that is, the current configuration information is obtained in real time and compared with the previous configuration information to obtain the change parameters, and then the data consumption thread group corresponding to the change parameters is adjusted in real time according to the change parameters, thereby realizing dynamic adjustment of consumption data on the data consumption system side, and improving the efficiency of data consumption thread adjustment and customer experience.

[0042] Figure 2 FIG. 1 is a flow chart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application. Figure 2 As shown, the above step 120 may include:

[0043] Step 121: extract the theme configuration information in the current configuration information and the theme configuration information in the previous configuration information.

[0044] In order to distinguish the types of data, data is usually divided into multiple topics according to different attributes such as type. After obtaining the current configuration information, the topic configuration information in the current configuration information and the topic configuration information in the previous configuration information are extracted, that is, the current configuration information and the previous configuration information are compared from a large category.

[0045] Step 122: Compare the theme configuration information in the current configuration information with the theme configuration information in the previous configuration information.

[0046] By comparing the subject configuration information in the current configuration information with the subject configuration information in the previous configuration information, the changes in the subject configuration information between the current configuration information and the previous configuration information are known, that is, the changes in the current configuration information and the previous configuration information are compared as a whole.

[0047] Correspondingly, step 130 includes:

[0048] Step 131: When the changed parameters include the first topic that exists in both the previous configuration information and the current configuration information, and there is a change between the first topic configuration information in the current configuration information and the first topic configuration information in the previous configuration information, restart the consumer thread group corresponding to the first topic.

[0049] When the first topic exists in both the previous configuration information and the current configuration information, and there is a change between the first topic configuration information in the current configuration information and the first topic configuration information in the previous configuration information, it means that the configuration information of the first topic (such as data format, data transmission frequency, field changes, etc.) has changed. At this time, it is necessary to restart the consumer thread group corresponding to the first topic and adjust the first topic configuration information to be consistent with the current configuration information.

[0050] Figure 3 is a flowchart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application. The first topic configuration information includes partition information; Figure 3 As shown, the above step 130 may also include:

[0051] Step 132: When there is a change between the data consumption thread group corresponding to the first topic in the previous configuration information and the data consumption thread group corresponding to the first topic in the current configuration information, adjust the partition information of the first topic configuration information in the current configuration information.

[0052] When there is a change between the data consumption thread group corresponding to the first topic in the previous configuration information and the data consumption thread group corresponding to the first topic in the current configuration information, it may cause the number of consumption thread groups in each partition corresponding to the first topic to change. In order to further improve the balance between the partitions of the first topic, when the number of consumption thread groups in each partition changes or the partition information changes, the partition information of the first topic configuration information is adjusted to ensure the balance of consumption thread groups between the partitions of the first topic (for example, try to make the number of thread groups in each partition equal or similar).

[0053] Figure 4 FIG. 1 is a flow chart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application. Figure 4 As shown, the above step 130 may include:

[0054] Step 133: When the changed parameters include configuration information of the second topic that exists in the previous configuration information and does not exist in the current configuration information, the consumer thread group corresponding to the second topic is stopped.

[0055] When the change parameter includes the configuration information of the second topic that exists in the previous configuration information but does not exist in the current configuration information, that is, the current data stream does not need the consumer thread group corresponding to the second topic. At this time, the consumer thread group corresponding to the second topic can be stopped to achieve a quick response.

[0056] Figure 5 FIG. 1 is a flow chart of a method for dynamically adjusting a data consumption thread provided by another exemplary embodiment of the present application. Figure 5As shown, the above step 130 may include:

[0057] Step 134: When the changed parameters include configuration information of the third topic that does not exist in the previous configuration information but exists in the current configuration information, start the consumption thread group corresponding to the third topic.

[0058] When the change parameter includes the configuration information of the third topic that does not exist in the last configuration information but exists in the current configuration information, that is, the current data stream needs the consumption thread group corresponding to the third topic. At this time, the consumption thread group corresponding to the third topic can be started immediately to achieve rapid response.

[0059] In practical applications, the above embodiment can be implemented by a work executor program, which establishes a message template processor proxy (KafkaTemplateProxy) to cache and manage the message template processor (KafkaTemplate) object corresponding to each topic (topic), and a streaming construction factory proxy class (StreamsBuilderFactoryBeanProxy) to cache and manage the streaming construction factory entity class (StreamsBuilderFactoryBean) object corresponding to each topic.

[0060] Specifically, the message template processor agent includes four program blocks: a registration program (register), an existence judgment program (isExist), a removal program (remove), and a handling program (handle). Among them, the registration program instantiates the message template processor object according to the configuration parameters, and caches the instantiated message template processor object in the memory with the consumer group id (primary key) as the key (establishment of the consumer thread group), the existence judgment program determines whether the message template processor object exists in the memory object (determines whether the corresponding thread group exists), the removal program removes the message template processor object in the memory object (when the corresponding thread group is not needed, delete the message template processor object), the execution program determines whether the configuration has changed, and implements the corresponding parameter comparison difference (the comparison operation in step 120).

[0061] The streaming factory proxy class inherits the lifecycle management (SmartLifecycle) and initializing entity (InitializingBean) interfaces. The streaming factory proxy class includes: start program (start), stop program (stop), and isRunning judgment program (isRunning). Among them, the start program builds the streaming factory entity class object according to the configuration parameters, caches the consumer group id (primary key) as the key in the memory object, and starts the corresponding thread group consumption. The stop program obtains the streaming factory entity class object in the memory, and then removes the cached object in the memory. The isRunning judgment program determines whether there is a corresponding consumer group cache in the memory. If there is such a cache, it is also necessary to determine whether the consumer group is running according to the cache object status. If it is running, it returns true (yes), if it is stopped, it returns false (no), and if there is no cache object, it directly returns false (no).

[0062] When the configuration changes, a message notification event will be sent. When the message template processor agent senses the parameter change, it will call the configuration center to obtain the latest configuration and call the execution program to compare the parameters of the previous version with the current parameters. The streaming construction factory agent class can start or stop the corresponding consumer thread group according to the corresponding parameter changes.

[0063] The present application also provides a method for adjusting the data consumption thread when the configuration parameters are changed in a non-running state, by configuring new configuration parameters on the configuration interface and publishing them in the form of events after the configuration is completed. After receiving the notification of the configuration parameter change, the latest configuration parameters are obtained, and the work executor (execute work) program is started to start the data consumption thread group corresponding to the new configuration parameters.

[0064] Exemplary Devices

[0065] Figure 6 FIG. 1 is a schematic diagram of a structure of a dynamic adjustment device for a data consumption thread provided by an exemplary embodiment of the present application. Figure 6 As shown, the dynamic adjustment device 60 includes: an acquisition module 61, used to obtain current configuration information; wherein the configuration information represents data consumption demand; a comparison module 62, used to compare the current configuration information with the previous configuration information to obtain a change parameter; wherein the change parameter is a parameter of the current configuration information that has changed relative to the previous configuration information; and an adjustment module 63, used to adjust the data consumption thread group corresponding to the change parameter in real time according to the change parameter.

[0066] The present application provides a device for dynamically adjusting data consumption threads, which obtains current configuration information representing data consumption needs through an acquisition module 61, and then a comparison module 62 compares the current configuration information with the previous configuration information to obtain a change parameter. Finally, an adjustment module 63 adjusts the data consumption thread group corresponding to the change parameter in real time according to the change parameter; that is, by obtaining the current configuration information in real time and comparing it with the previous configuration information to obtain the change parameter, and then adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter, thereby realizing dynamic adjustment of consumption data on the data consumption system side, and improving the efficiency of data consumption thread adjustment and customer experience.

[0067] In one embodiment, the acquisition module 61 may be further configured to: acquire current configuration information when a broadcast event representing adjustment configuration information is received.

[0068] Figure 7 FIG. 1 is a schematic diagram of a structure of a dynamic adjustment device for a data consumption thread provided by another exemplary embodiment of the present application. Figure 7 As shown, the comparison module 62 may include: a theme extraction unit 621, used to extract theme configuration information in the current configuration information and theme configuration information in the previous configuration information; a theme comparison unit 622, used to compare theme configuration information in the current configuration information and theme configuration information in the previous configuration information. Correspondingly, the adjustment module 63 may be further configured as follows: when the change parameter includes the first theme that exists in both the previous configuration information and the current configuration information, and there is a change between the first theme configuration information in the current configuration information and the first theme configuration information in the previous configuration information, restart the consumer thread group corresponding to the first theme.

[0069] In one embodiment, the adjustment module 63 can be further configured to: when there is a change between the data consumption thread group corresponding to the first topic in the previous configuration information and the data consumption thread group corresponding to the first topic in the current configuration information, adjust the partition information of the first topic configuration information in the current configuration information.

[0070] In one embodiment, the adjustment module 63 may be further configured to: when the changed parameters include configuration information of the second topic that exists in the previous configuration information and does not exist in the current configuration information, stop the consumer thread group corresponding to the second topic.

[0071] In one embodiment, the adjustment module 63 may be further configured to: when the changed parameters include configuration information of a third theme that does not exist in the previous configuration information but exists in the current configuration information, start the consumption thread group corresponding to the third theme.

[0072] Exemplary Electronic Devices

[0073] Below, reference Figure 8 The electronic device according to the embodiment of the present application is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signal from them.

[0074] Figure 8 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0075] like Figure 8 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0076] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0077] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the dynamic adjustment method of the data consumption thread of each embodiment of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0078] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0079] When the electronic device is a stand-alone device, the input device 13 may be a communication network connector, which is used to receive the collected input signals from the first device and the second device.

[0080] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0081] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0082] Of course, to simplify, Figure 8 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0083] Exemplary computer program products and computer-readable storage media

[0084] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the dynamic adjustment method of data consumption threads according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0085] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0086] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method for dynamically adjusting data consumption threads according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0087] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0088] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0089] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0090] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0091] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0092] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for dynamically adjusting a data consumption thread, characterized in that: include: Acquire current configuration information; wherein the configuration information represents data consumption requirements; Comparing the current configuration information with the last configuration information to obtain a change parameter; wherein the change parameter is a parameter of the current configuration information that has changed relative to the last configuration information; and According to the change parameter, adjusting the data consumption thread group corresponding to the change parameter in real time; The comparing the current configuration information with the last configuration information comprises: Extracting the theme configuration information in the current configuration information and the theme configuration information in the previous configuration information; and Comparing the theme configuration information in the current configuration information with the theme configuration information in the previous configuration information; The step of adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter includes: When the change parameter includes a first topic that exists in both the previous configuration information and the current configuration information, and there is a change between the first topic configuration information in the current configuration information and the first topic configuration information in the previous configuration information, restarting the consumer thread group corresponding to the first topic; The restarting of the consumer thread group corresponding to the first topic includes: Adjust the first topic configuration information according to the current configuration information, and restart the consumer thread group corresponding to the first topic; The first topic configuration information includes partition information; wherein, adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter also includes: When there is a change between the data consumption thread group corresponding to the first topic in the previous configuration information and the data consumption thread group corresponding to the first topic in the current configuration information, the partition information of the first topic configuration information in the current configuration information is adjusted.

2. The dynamic adjustment method according to claim 1, characterized in that: The obtaining of current configuration information includes: When a broadcast event indicating adjustment of the configuration information is received, the current configuration information is acquired.

3. The dynamic adjustment method according to claim 1, characterized in that: The step of adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter comprises: When the change parameter includes the second theme configuration information that exists in the previous configuration information and does not exist in the current configuration information, the consumer thread group corresponding to the second theme configuration information is stopped.

4. The dynamic adjustment method according to claim 1, characterized in that: The step of adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter comprises: When the change parameter includes third theme configuration information that does not exist in the previous configuration information but exists in the current configuration information, the consumer thread group corresponding to the third theme configuration information is started.

5. A dynamic adjustment device for a data consumption thread, characterized in that: include: An acquisition module, used to acquire current configuration information; wherein the configuration information represents data consumption requirements; A comparison module, used to compare the current configuration information with the last configuration information to obtain a change parameter; wherein the change parameter is a parameter of the current configuration information that has changed relative to the last configuration information; The comparing the current configuration information with the last configuration information comprises: Extracting the theme configuration information in the current configuration information and the theme configuration information in the previous configuration information; and Comparing the theme configuration information in the current configuration information with the theme configuration information in the previous configuration information; and An adjustment module, used for adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter; The step of adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter includes: When the change parameter includes a first topic that exists in both the previous configuration information and the current configuration information, and there is a change between the first topic configuration information in the current configuration information and the first topic configuration information in the previous configuration information, restarting the consumer thread group corresponding to the first topic; The restarting of the consumer thread group corresponding to the first topic includes: Adjust the first topic configuration information according to the current configuration information, and restart the consumer thread group corresponding to the first topic; The first topic configuration information includes partition information; wherein, adjusting the data consumption thread group corresponding to the change parameter in real time according to the change parameter also includes: When there is a change between the data consumption thread group corresponding to the first topic in the previous configuration information and the data consumption thread group corresponding to the first topic in the current configuration information, the partition information of the first topic configuration information in the current configuration information is adjusted.

6. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method for dynamically adjusting data consumption threads according to any one of claims 1 to 4.

7. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to execute the dynamic adjustment method of the data consumption thread described in any one of claims 1-4 above.

Images