Message filtering method and device, computer device, medium and product

Abstract

The application relates to a message filtering method and device, computer equipment, medium and product. The method comprises the following steps: in response to a message extraction operation, determining a consumer group corresponding to the message extraction operation; calling a filter matched with the consumer group to obtain each subscription message subscribed by the consumer group from a message cluster; the consumer end is configured with multiple filters; the subscription message is produced by a producer and published in the message cluster; for each subscription message, according to a message type of the subscription message, determining selected filtering conditions of the subscription message from candidate filtering conditions configured by the filter; and filtering the subscription message in the case that the subscription message meets the selected filtering conditions. The method can automatically filter the message, and improves the accuracy and pertinence of the message acquisition of the consumer.

Description

Technical Field

[0001] This application relates to the field of distributed message processing technology, and in particular to a message filtering method, apparatus, computer device, storage medium, and computer program product. Background Technology

[0002] Distributed messaging systems, such as Kafka, are distributed stream processing platforms used for real-time processing and storage of data streams. Due to the advantages of distributed messaging systems, such as visual cluster management and integrated activation, they are widely used in business development in enterprises or organizations.

[0003] Enterprises or organizations often provide self-built message clusters on their basic platforms for use by various internal technical teams. The downstream systems of these technical teams are essentially consumers in a distributed messaging system. Within an enterprise or organization, multiple consumers may simultaneously require data from a single system (such as orders), resulting in a massive amount of data. However, each consumer client only needs to focus on data relevant to its own system. Therefore, in existing technologies, after receiving subscribed messages, each consumer must manually sift through the massive amount of messages to extract the data it needs. This leads to untimely fulfillment of consumers' business demands and reduces the R&D efficiency of the enterprise or organization's business development. Summary of the Invention

[0004] Therefore, it is necessary to provide a message filtering method, device, computer equipment, computer-readable storage medium, and computer program product that can automatically filter messages, improve the accuracy and relevance of consumer message acquisition, and thus improve the business development speed, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a message filtering method applied to a consumer, the method comprising:

[0006] In response to a message retrieval operation, determine the consumer group corresponding to the message retrieval operation;

[0007] The consumer group's matching filter is invoked to retrieve the subscription messages subscribed to by the consumer group from the message cluster; the consumer is configured with multiple filters; the subscription messages are produced by the producer and published in the message cluster.

[0008] For each of the subscription messages, the selected filtering conditions for the subscription message are determined from the candidate filtering conditions configured in the filter, based on the message type of the subscription message.

[0009] If the subscribed message meets the selected filtering conditions, the subscribed message is filtered.

[0010] In some embodiments, the number of selected filtering conditions is multiple, and filtering the subscription message when the subscription message satisfies the selected filtering conditions includes:

[0011] Extract the target filtering condition from each of the selected filtering conditions;

[0012] The subscription message is filtered and detected based on the target filtering conditions to determine the filtering detection result of the subscription message and the target filtering conditions;

[0013] If the filtering detection result indicates that the subscription message does not meet the target filtering condition, the target filtering condition is extracted from the remaining selected filtering conditions, and the step of filtering the subscription message based on the target filtering condition is returned.

[0014] If the filtering detection result indicates that the subscription message meets the target filtering condition, the subscription message is filtered.

[0015] In some embodiments, retrieving the target filter condition from each of the selected filter conditions includes:

[0016] Obtain the expected detection time for each of the selected filtering conditions;

[0017] Select the target filter condition with the shortest expected detection time from the selected filter conditions.

[0018] In some embodiments, the method further includes:

[0019] In response to a condition addition instruction sent by the message cluster control platform, the system retrieves the new filtering condition and the condition configuration information of the new filtering condition from the cache database of the message cluster control platform.

[0020] Based on the condition configuration information, a target filter for configuring the new filtering conditions is determined from each of the filters;

[0021] The newly added filter condition is cached in the local cache area of ​​the target filter, and the newly added filter condition is configured in the target filter.

[0022] In some embodiments, the condition configuration information includes the type of the new condition for the new filter condition; the method further includes:

[0023] The type of the newly added filter condition is matched with the type of the filter condition of each candidate filter condition in the target filter.

[0024] If a filter condition type that matches the newly added condition type exists, then a filter condition of the same type as the newly added filter condition is determined from each of the candidate filter conditions.

[0025] If the newly added filter condition is repeatedly verified against the existing filter conditions of the same type, and the verification is successful, it is determined that the newly added filter condition will not be configured.

[0026] In some embodiments, the method further includes:

[0027] Obtain the failure criteria for each of the candidate filtering conditions in the filter;

[0028] For each candidate filtering condition, an effectiveness test is performed on the candidate filtering condition according to the failure criteria to determine the effectiveness test result of the candidate filtering condition;

[0029] If the validity detection result indicates that the candidate filtering condition has failed, the candidate filtering condition is removed from the local cache area of ​​the filter.

[0030] Secondly, this application also provides a message filtering device, the device comprising:

[0031] An operation response module is used to respond to a message extraction operation and determine the consumer group corresponding to the message extraction operation.

[0032] The filter invocation module is used to invoke the filters matched by the consumer group to obtain the subscription messages subscribed to by the consumer group from the message cluster; the consumer is configured with multiple filters; the subscription messages are produced by the producer and published in the message cluster;

[0033] The selected filter condition determination module is used to determine the selected filter condition for each subscription message from the candidate filter conditions configured in the filter, based on the message type of the subscription message.

[0034] The message filtering module is used to filter the subscribed message if the subscribed message meets the selected filtering conditions.

[0035] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0037] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0038] The aforementioned message filtering methods, devices, computer equipment, storage media, and computer program products are configured with multiple filters on the consumer side. Each filter is matched with a consumer group on the consumer side. When a consumer group triggers a message extraction operation, the consumer side can respond to the message extraction operation, determine the consumer group corresponding to the message extraction operation, call the filter matching the message group, and retrieve the subscription messages subscribed to by the consumer group from the message cluster. For each subscription message, the consumer side can determine the selected filtering condition for the subscription message from the candidate filtering conditions configured in the filter according to the message type of the subscription message. The selected filtering condition is used to filter and detect the subscription messages. From the numerous subscription messages, the subscription messages that meet the selected filtering condition, i.e., the subscription messages that the consumer group does not need, are filtered in advance. This can effectively improve the accuracy and relevance of messages obtained by each consumer in the consumer group. Each consumer only needs to focus on the messages it needs, thereby improving the business development speed. Attached Figure Description

[0039] Figure 1 These are schematic diagrams of the structure of existing distributed messaging systems in some embodiments;

[0040] Figure 2 This is a schematic diagram of the structure of a distributed messaging system in some other embodiments;

[0041] Figure 3 This is a flowchart illustrating the message filtering method in some embodiments;

[0042] Figure 4 This is a flowchart illustrating the process of filtering subscription messages when the subscription messages meet the selected filtering conditions in some embodiments.

[0043] Figure 5 This is a flowchart illustrating the message filtering method in some other embodiments;

[0044] Figure 6 This is a flowchart illustrating the message filtering method in some other embodiments;

[0045] Figure 7 This is a flowchart illustrating the message filtering method in some other embodiments;

[0046] Figure 8 This is a schematic diagram of the structure of a distributed messaging system in some other embodiments;

[0047] Figure 9 This is a flowchart illustrating the message filtering method in some other embodiments;

[0048] Figure 10 This is a structural block diagram of the message filtering device in some embodiments;

[0049] Figure 11 This is a diagram showing the internal structure of a computer device in some embodiments. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In some embodiments, the structure of an existing distributed messaging system, such as Kafka, is as follows: Figure 1 As shown, the Kafka system is a high-throughput distributed publish-subscribe messaging system. The system as a whole consists of a producer (101), a message cluster (102), and a consumer (103).

[0052] Among them, the production end 101 includes multiple producers 1011, and the producers 1011 are responsible for producing and publishing messages to the message cluster 102.

[0053] The message cluster 102 includes one or more servers 1021. Messages published by producer 1011 will be stored on server 1021. Each message published to the message cluster 102 has a category, which is called Topic within the message cluster. Physically, messages of different Topics will be stored separately, while logically, messages of one Topic can be stored on one or more servers 1021.

[0054] The consumer end 103 includes multiple consumer groups 1031, and each consumer group 1031 contains multiple consumers 1032. Consumer 1032 refers to message consumers, which are clients that can read messages from message cluster 102. Each consumer 1032 belongs to a specific consumer 1031.

[0055] In the existing Kafka system, as shown by the arrows in the diagram, after producer 1011 produces a message, it publishes the message to message cluster 102. Multiple servers 1021 in message cluster 102 store messages for different topics. When consumer 1032 needs to consume messages, it can retrieve the subscription messages corresponding to the subscribed topic from each server 1021 in consumer cluster 102. Since the Kafka system does not have message filtering capabilities, consumer 1032 will obtain all messages for the subscribed topic and then filter them manually. This greatly reduces the efficiency of consumers in obtaining effective messages, and consumers' business needs cannot be met in a timely manner, which will seriously reduce the efficiency of business development.

[0056] Based on this, such as Figure 2 As shown, this application provides a distributed messaging system, which includes a producer 101, a consumer cluster 102, and a consumer 201. Unlike the traditional Kafka system, the consumer 201 in this application includes not only multiple consumer groups 1031 and multiple consumers 1032 in each consumer group 1031, but also filters 2011 set corresponding to each consumer group 1031.

[0057] In practice, consumer 201 responds to a message retrieval operation by determining the corresponding consumer group 1031, invoking the filter 2011 that matches consumer group 1031, and retrieving all subscribed messages from message cluster 102. For each subscribed message, based on its message type, the selected filtering condition is determined from the candidate conditions configured in the filter. If the subscribed message meets the selected filtering condition, it is filtered. By configuring a corresponding filter 2011 for each consumer group 1031, the accuracy and relevance of message retrieval by each consumer in the group can be effectively improved. Each consumer only needs to focus on the messages it requires, thereby increasing the business development speed.

[0058] In some embodiments, such as Figure 3 As shown, a message filtering method is provided, which can be applied to... Figure 2 Taking the consumer side 201 as an example, the explanation includes the following steps:

[0059] S302, in response to a message retrieval operation, determines the consumer group corresponding to the message retrieval operation.

[0060] Message retrieval operation refers to the process by which a consumer retrieves messages from a message cluster. Through this operation, the consumer can obtain pre-subscribed message data from the message cluster. Understandably, message retrieval operations can be triggered in two modes: pull mode, where the consumer periodically polls the message cluster, sending pull requests to the servers within the cluster to obtain new message data; and push mode, where the consumer cluster can push the latest subscribed messages to the consumer in real time. Therefore, the message retrieval operation can be triggered by the client corresponding to the consumer or by the message cluster itself. However, regardless of who triggers the operation, it will always correspond to a consumer group, ensuring that the corresponding subscribed messages can be pushed or retrieved to the appropriate consumer group.

[0061] Specifically, the consumer responds to the message retrieval operation by determining the consumer group corresponding to the message retrieval operation based on the consumer group identifier carried in the operation information of the message retrieval operation.

[0062] S304 invokes the filter matching the consumer group to retrieve the subscription messages of each consumer group from the message cluster.

[0063] Filters are components used to filter messages subscribed to by a consumer group. Multiple filters are configured on the consumer side, each corresponding to its own consumer group. Each filter only provides message filtering operations for its designated consumer group. Filters are configured with multiple candidate filtering conditions to filter invalid messages for that specific consumer group. Understandably, filters can be pre-designed and integrated into the consumer side.

[0064] In some embodiments, multiple candidate filtering conditions in the filter can be generated and configured in the filter by the designer according to the actual filtering needs of the consumer.

[0065] In some embodiments, multiple candidate filtering conditions in the filter can be obtained and dynamically configured by the filter according to the actual filtering needs of the consumer.

[0066] Consumer groups can subscribe to messages of multiple categories. When extracting messages, the filter retrieves all subscribed messages of each category from the message cluster. For example, if consumer group A subscribes to messages of Topic A and Topic B, the filter will retrieve all updated subscription messages of Topic A and Topic B from the message cluster. In each subscription message, in addition to the valid messages required by consumer group A, there will also be invalid messages, that is, valid messages required by other consumer groups that have subscribed to the same category, other than consumer group A.

[0067] Specifically, after determining the consumer group corresponding to the consumption retrieval operation, the consumer can determine the filter that matches the consumer group, call the filter that matches the consumer group, and retrieve the subscription messages subscribed to by the consumer group from the message cluster.

[0068] S306, For each subscribed message, determine the selected filtering condition for the subscribed message from the candidate filtering conditions configured in the filter, based on the message type of the subscribed message.

[0069] Among them, the message type of the subscription message is a type parameter used to classify each subscription message. Taking business subscription messages as an example, message types can include business orders, business topics, payment messages, shipping messages, etc. Different message types can use different candidate filtering conditions.

[0070] Specifically, for each subscribed message, the consumer can determine the selected filtering condition for the subscribed message from the candidate filtering conditions of the filter based on the message type of the subscribed message.

[0071] In some embodiments, the filter pre-sets a correspondence between message types and candidate filter conditions. The consumer can look up the correspondence between message types and candidate filter conditions based on the message type of the subscribed message, determine the candidate filter condition that matches the message type of the subscribed message, and set the candidate filter condition as the selected filter condition for the subscribed message. It is understood that one message type corresponds to at least one candidate filter condition.

[0072] In some embodiments, the filter pre-selects candidate filter condition types and establishes a correspondence between them and the message types of the subscribed messages. The consumer can then look up the correspondence between message types and filter condition types based on the message types of the subscribed messages, determine the filter condition type that matches the message type of the subscribed messages, and then determine the candidate filter conditions based on the filter condition types. Here, the filter condition type is a type parameter used to distinguish the filtering methods used by each candidate filter condition. It can be understood that each filter condition type corresponds to at least one candidate filter condition.

[0073] S308: Filter the subscribed messages if they meet the selected filtering conditions.

[0074] Specifically, the consumer can filter and check the subscribed messages based on the selected filtering conditions to determine whether the subscribed messages meet the selected filtering conditions. If the subscribed messages meet the selected filtering conditions, they can be determined to be invalid message data relative to the consumer group. The consumer can then filter the subscribed messages, that is, discard the subscribed messages and not push them.

[0075] In some embodiments, if the subscribed message does not meet the selected filtering conditions, the consumer can determine that the subscribed message is valid message data relative to the consumer group and can push the subscribed message.

[0076] In the aforementioned message filtering method, multiple filters are configured on the consumer side. Each filter matches a consumer group on the consumer side. When a consumer group triggers a message extraction operation, the consumer side can respond to the message extraction operation, determine the consumer group corresponding to the message extraction operation, call the filter matching the message group, and retrieve the subscription messages subscribed to by the consumer group from the message cluster. For each subscription message, the consumer side can determine the selected filtering condition for the subscription message from the candidate filtering conditions configured in the filter according to the message type of the subscription message. The selected filtering condition is used to filter and detect the subscription messages. From the numerous subscription messages, the subscription messages that meet the selected filtering condition, i.e., the subscription messages that the consumer group does not need, are filtered in advance. This can effectively improve the accuracy and relevance of messages obtained by each consumer in the consumer group. Each consumer only needs to focus on the messages it needs, thereby improving the business development speed.

[0077] Understandably, different filter condition types for selected filtering conditions in subscribed messages require different filtering detection methods. For example, in some embodiments, when the selected filtering condition type is a numerical detection type, the consumer can obtain the filter parameters of the selected filtering condition and the parameter range corresponding to the filter parameters. Based on the filter parameters, the consumer can extract the parameter values ​​corresponding to the filter parameters from the subscribed message. If the parameter values ​​are within the parameter range, the consumer can determine that the subscribed message meets the selected filtering condition.

[0078] In other embodiments, when the filter condition type of the selected filter condition is keyword detection type, the consumer can obtain the comparison keyword in the selected condition and the position information of the comparison keyword in the subscription information. Based on the position information of the comparison keyword in the subscription information, the comparison information to be compared is obtained from the subscription information. The comparison information to be compared is compared with the comparison keyword. If the comparison is inconsistent, it is determined that the subscription message meets the selected filter condition.

[0079] In some embodiments, a subscription message of a consumption type can correspond to multiple selected filtering conditions. Taking a subscription message as a payment message as an example, the corresponding selected filtering conditions can be filtering conditions based on timestamps or filtering conditions based on keywords.

[0080] Based on this, in some embodiments, such as Figure 4 As shown, multiple filtering conditions can be selected. In S308, if a subscribed message meets the selected filtering conditions, the subscribed message is filtered, including:

[0081] S402, extract the target filter condition from the selected filter conditions.

[0082] Specifically, when multiple selected filtering conditions exist, the consumer can extract the target filtering condition from these multiple selected filtering conditions.

[0083] In some embodiments, the consumer can randomly select one of the selected filtering conditions as the target filtering condition.

[0084] S404, Perform filtering detection on the subscribed messages based on the target filtering conditions, and determine the filtering detection results of the subscribed messages and the target filtering conditions.

[0085] Specifically, the consumer can perform filtering detection on the subscribed messages based on the defined target filtering conditions, and determine the filtering detection results of the subscribed messages and the target filtering conditions.

[0086] S406, if the filtering detection result indicates that the subscribed message does not meet the target filtering condition, extract the target filtering condition from the remaining selected filtering conditions and return to the step of filtering the subscribed message based on the target filtering condition.

[0087] Specifically, if the filtering detection result indicates that the subscribed message does not meet the target filtering condition, the consumer can extract the target filtering condition from the remaining selected filtering conditions and return the step of filtering the subscribed message based on the target filtering condition.

[0088] S408, If the filtering detection results indicate that the subscribed messages meet the target filtering conditions, then the subscribed messages are filtered.

[0089] Specifically, the consumer repeatedly executes the operation of retrieving the target filtering condition and filtering the subscribed messages based on the target filtering condition until a filtering detection result of the target filtering condition indicates that the subscribed message meets the target filtering condition. At this point, the consumer can determine that the subscribed message is invalid message data relative to the consumer group and filter the subscribed message.

[0090] In the above embodiments, when there are multiple selected filtering conditions for the subscribed message, the consumer performs filtering detection by cyclically extracting the target filtering conditions. The subscribed message is filtered if it meets any one of the target filtering conditions, instead of performing all the selected filtering conditions sequentially. This can improve the detection accuracy of filtering detection while effectively improving the detection efficiency.

[0091] In addition to random sampling, in order to further improve detection efficiency, in some embodiments, S402, the target filter condition is extracted from each selected filter condition, including: obtaining the expected detection time of each selected filter condition, and extracting the target filter condition with the shortest expected detection time from each selected filter condition.

[0092] The expected detection duration of the selected filtering conditions is the predicted detection duration when filtering any subscribed message using the selected filtering conditions. The predicted detection duration can be determined by the designer based on the experimental or empirical duration when filtering using the selected filtering conditions.

[0093] Specifically, when the consumer determines that there are multiple selected filtering conditions for the subscribed message, it can obtain the expected detection duration of each selected filtering condition. Based on the expected detection duration, the selected filtering condition with the shortest expected detection duration is selected as the target filtering condition. Subsequently, the subscribed message is filtered and detected based on the target filtering condition.

[0094] In the above embodiments, by selecting the target filter condition with the shortest expected detection time from each selected filter condition according to its expected detection time, it can be ensured that the filter condition used each time a filter detection is performed is the filter condition with the shortest expected detection time among the remaining selected filter conditions, which effectively improves the detection efficiency of filtering and detecting subscription information.

[0095] In some embodiments, such as Figure 5 As shown, the message filtering method also includes:

[0096] S502, in response to the condition addition instruction sent by the message cluster control platform, retrieves the new filtering condition and the condition configuration information of the new filtering condition from the cache database of the message cluster control platform.

[0097] Among them, the new filter conditions are the filter conditions that consumers need to add to the filter based on their actual message consumption needs. The condition configuration information of the new filter conditions is information data used to reflect the configuration of the new filter conditions. For example, the condition configuration information may include the configuration filter identifier of the new filter condition, the type of the new condition, the types of messages that can be filtered, etc.

[0098] The message cluster control platform is used to manage and monitor message clusters, as well as to configure, deploy, and maintain them to ensure the stability of the cluster and distributed messaging system. Consumers can configure filters through the message cluster control platform.

[0099] Among them, the condition addition instruction is a signal used to prompt the consumer that there is a filter that needs to be added. The condition addition instruction can be sent from the message cluster control platform to the consumer.

[0100] In some embodiments, consumers can log in to the message cluster control platform via a client, generate new filter conditions in the platform, and the consumer cluster control platform obtains the generated new filter conditions and their corresponding configuration information. The platform then caches the new filter conditions and their configuration information in a cache database and sends a condition addition command to the consumer. The cache database can be a Remote Dictionary Server (Redis) database.

[0101] Specifically, in response to the condition addition command sent by the message cluster control platform, the consumer retrieves the latest cached new filtering conditions and the corresponding condition configuration information from the message cluster control platform's cache database. It is understood that the number of new filtering conditions can be one or more, and this application does not limit this.

[0102] S504, based on the condition configuration information, determines the target filter from each filter for configuring new filter conditions.

[0103] Specifically, after receiving the condition configuration information for the new filtering conditions, the consumer can determine the filter whose filter identifier matches the configuration filter identifier from among all filters based on the configuration filter identifier in the condition configuration information, and use it as the target filter for configuring the new filtering conditions.

[0104] S506, the newly added filter conditions are cached in the local cache area of ​​the target filter, and the newly added filter conditions are configured in the target filter.

[0105] The local cache area is the cache area corresponding to the target filter. By caching the new filter conditions to the local cache area corresponding to the target filter, the filter can be effectively isolated from other filters, thereby improving the accuracy of filter message filtering and the efficiency of filter detection.

[0106] Specifically, after the consumer determines the target filter for configuring new filtering conditions, it can cache the new filtering conditions in the local cache area of ​​the target filter to complete the configuration of the new filtering conditions in the target filter.

[0107] In some embodiments, after the consumer caches the new filter conditions to the local cache area of ​​the target filter, it also needs to update the correspondence between message types and candidate filter conditions in the target filter according to the filterable message types of the new message types, so as to complete the configuration of the new filter conditions in the target filter.

[0108] In the above embodiments, the consumer can respond to the condition addition command sent by the message cluster control platform at any time, obtain the corresponding new filtering conditions from the cache database, and update the filter conditions. This enables the filter to achieve dynamic configuration of conditions, improves the adaptability of the filter in the changing filtering and inspection environment, and meets the various message filtering needs of consumers.

[0109] Furthermore, in some embodiments, such as Figure 6 As shown, the condition configuration information includes the new condition type for adding filter conditions, and the message filtering method also includes:

[0110] S602, Match the type of the new condition of the newly added filter condition with the type of the filter condition of each candidate filter condition in the target filter.

[0111] Among them, the newly added condition type is the condition type corresponding to the newly added filter condition.

[0112] Specifically, after obtaining the condition configuration information for the new filter condition, the consumer can obtain the new condition type from the condition configuration information, and at the same time obtain the filter condition type of each candidate filter condition in the target filter. The new condition type is then matched with each filter condition type to determine whether there is a condition type in each filter condition type that matches the new condition type.

[0113] S604 If there exists a filter condition type that matches the type of the newly added condition, then determine the filter condition of the same type as the newly added filter condition from each candidate filter condition.

[0114] Specifically, if it is determined that there is a filter condition type among the various filter condition types that matches the new condition type, it means that there may be already configured filter conditions in the target filter that are duplicates of the new filter condition. The consumer can then identify the candidate filter conditions corresponding to the filter condition type that matches the new condition type as filter conditions of the same type as the new filter condition. It is understood that the new filter condition can have one or more filter conditions of the same type.

[0115] S606: If the newly added filter condition is repeatedly verified against the same type of filter condition, and the verification is successful, it is determined that no new filter condition will be configured.

[0116] Among them, duplicate verification is a verification operation used to check whether a newly added filter condition is a duplicate of a filter condition of the same type.

[0117] Specifically, after the consumer determines the same type of filter condition for the new filter condition, it can perform duplicate verification between the new filter condition and the same type of filter condition. If there is a same type of filter condition that has been successfully verified as the new filter condition, it is determined that the new filter condition is duplicated with the same type of filter condition, and the consumer will determine not to configure the new filter condition.

[0118] In some embodiments, a duplicate verification model can be pre-configured on the consumer side. After determining the same type of filter condition for the new filter condition, the new filter condition and the same type of filter condition can be directly input into the duplicate verification model for duplicate verification to determine the duplicate verification result of the new filter condition and the same type of filter condition.

[0119] In some embodiments, the consumer can compare the new filtering conditions with the same type of filtering conditions to determine the condition similarity between the new filtering conditions and the same type of filtering conditions. If the condition similarity is greater than a preset similarity threshold, it is determined that the new filtering conditions and the same type of filtering conditions have been successfully verified.

[0120] In some embodiments, if the consumer determines that a newly added filter condition is duplicated with a similar filter condition, it can generate a duplicate condition warning message. This message indicates that a configured filter condition exists that duplicates the newly added filter condition. The consumer can then send this message to the message cluster control platform. Upon receiving the message, the platform can display the message to the consumer.

[0121] In the above embodiments, by repeatedly verifying the new filtering conditions before configuring them in the target filter, the probability of configuring duplicate filtering conditions in the filter can be effectively reduced.

[0122] In some embodiments, such as Figure 7 As shown, the message filtering method also includes:

[0123] S702, obtain the failure criteria for each candidate filtering condition in the filter.

[0124] The failure criteria for candidate filtering conditions are preset standards used to determine whether a candidate filtering condition has failed. Understandably, different candidate filtering conditions may have different failure criteria; for example, the failure criteria could be that the configured duration has reached a preset duration threshold, or that a removal command for the candidate filtering condition has been received.

[0125] Specifically, for each filter, the consumer can obtain the failure criteria for each candidate filtration condition in the filter, so as to provide a basis for subsequent failure detection of the candidate filtration conditions.

[0126] S704. For each candidate filtering condition, the effectiveness of the candidate filtering condition is tested according to the failure criteria of the candidate filtering condition, and the effectiveness test result of the candidate filtering condition is obtained.

[0127] Validity detection is an operation used to determine whether candidate filtering conditions have failed. Understandably, the specific steps for validity detection differ depending on the failure criteria. For example, when the failure criterion is that the configuration duration has reached a preset duration threshold, the consumer can obtain the configuration time node of the candidate filtering condition, determine the configuration duration of the candidate filtering condition based on the configuration time node and the current time node, and compare the configuration duration with the preset duration threshold to complete the validity detection of the candidate filtering condition. As another example, when the failure criterion is receiving a removal command for a candidate filtering condition, the consumer can determine whether the message cluster control platform has sent a removal command for the candidate filtering condition, thereby completing the validity detection of the candidate filtering condition.

[0128] Specifically, for each candidate filtering condition, the consumer can perform an effectiveness test on the candidate filtering condition based on the failure criteria of the candidate filtering condition, and obtain the effectiveness test result of the candidate filtering condition.

[0129] S706, if the validity test result indicates that the candidate filter condition has failed, remove the candidate filter condition from the local cache area of ​​the filter.

[0130] Specifically, if the validity test result of the candidate filtering condition indicates that the candidate filtering condition has failed, the consumer can determine that the candidate filtering condition has failed. In order to reduce the possibility that the consumer will lose valid information due to the use of failed filter submission for message filtering, the consumer can remove the candidate filtering condition from the local cache area of ​​the target filter to remove the candidate filtering condition from the filter configuration.

[0131] In some embodiments, when the validity detection result of the candidate filtering condition is that the configuration duration of the candidate filtering condition has reached a preset duration threshold, it is determined that the candidate filtering condition has failed.

[0132] In some embodiments, when the validity detection result of the candidate filtering condition is that the consumer receives a removal instruction sent by the consumer cluster control platform for the candidate filtering condition, it is determined that the candidate filtering condition has become invalid.

[0133] In the above embodiments, by using the failure criteria of candidate filtering conditions to detect the validity of candidate filtering conditions, invalid candidate filtering conditions can be removed from the filter in a timely manner, which effectively reduces the possibility that consumers will lose valid information due to the use of invalid filtering submissions for message filtering, and improves the accuracy of message filtering.

[0134] In some embodiments, a message filtering method is provided, which is applied to, for example, Figure 8 Taking the Kafka system shown as an example, the Kafka system includes a producer, a consumer cluster, a consumer, a message cluster control platform, and a cache database. The producer includes multiple producers, the consumer cluster has multiple servers, and the consumer includes multiple consumer groups and filters corresponding to each consumer group.

[0135] like Figure 9 As shown, the method includes the following steps:

[0136] Firstly, consumers can configure filtering conditions on the consumer cluster control platform via their client. The consumer cluster control platform responds to the consumer's condition configuration operation by obtaining the candidate filtering conditions for the consumer group to which the consumer belongs, as well as the condition configuration information for these candidate filtering conditions. This condition configuration information includes the configuration filter identifier, filtering condition type, and filterable message types for each candidate filtering condition.

[0137] The consumer cluster control platform writes the candidate filtering conditions and their configuration information into the cache database, generates a condition addition instruction, and sends the condition addition instruction to the consumer.

[0138] The consumer responds to the condition addition command sent by the consumer cluster control platform by retrieving candidate filtering conditions and their configuration information from the platform's cache database. Based on the configuration information, it determines the target filter from among the filters used to configure the candidate filtering conditions.

[0139] The consumer generates a mapping between candidate filtering conditions and message types based on the candidate filtering conditions and their configuration information. It then stores the candidate filtering conditions, configuration information, and mapping between candidate filtering conditions and message types in the local cache area of ​​the target filter, and completes the configuration of candidate filtering conditions in the target filter.

[0140] In practical use, the consumer can respond to a message retrieval operation, determine the corresponding consumer group, invoke the filter matching the consumer group, and retrieve the subscribed messages from the message cluster. For each subscribed message, the consumer can determine the selected filter condition matching the message type based on the message type of the subscribed message and the correspondence between the candidate filter conditions configured in the filter and the message type.

[0141] When only one filter condition is selected, the consumer performs filtering checks on the subscribed messages based on the selected filter condition. If the subscribed message meets the selected filter condition, it is filtered; otherwise, it is pushed to the corresponding consumer group.

[0142] When multiple filtering conditions are selected, the consumer can obtain the expected detection duration for each selected filtering condition. Based on these expected detection durations, the target filtering condition with the shortest expected detection duration is selected. The subscription message is then filtered using this target filtering condition. The filtering result between the subscription message and the target filtering condition is determined. If the filtering result indicates that the subscription message does not meet the target filtering condition, the target filtering condition is selected from the remaining selected filtering conditions, and the process of filtering the subscription message based on this target filtering condition is returned. If the filtering result indicates that the subscription message meets the target filtering condition, the subscription message is filtered. Once all selected filtering conditions have been tested, and the filtering results all indicate that the subscription message does not meet the target filtering condition, the subscription message is pushed to the corresponding consumer group.

[0143] The message filtering method in this embodiment filters the messages to be delivered before they are delivered to the consumer, allowing the consumer client to focus only on the messages it needs. This improves the accuracy and relevance of the messages received by the consumer. Furthermore, the filter supports dynamic configuration of filtering conditions, allowing each business system to configure corresponding filtering conditions to filter message data according to its own filtering needs. This reduces maintenance costs and improves system processing efficiency with zero code changes.

[0144] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0145] Based on the same inventive concept, this application also provides a message filtering device for implementing the message filtering method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more message filtering device embodiments provided below can be found in the limitations of the message filtering method described above, and will not be repeated here.

[0146] In some embodiments, such as Figure 10As shown, a message filtering device 1000 is provided, including: an operation response module 1001, a filter invocation module 1002, a selected filter condition determination module 1003, and a message filtering module 1004, wherein:

[0147] The operation response module 1001 is used to respond to a message extraction operation and determine the consumer group corresponding to the message extraction operation.

[0148] The filter call module 1002 is used to call the filter matched by the consumer group to obtain the subscription messages subscribed to by the consumer group from the message cluster; the consumer is configured with multiple filters; the subscription messages are produced by the producer and published in the message cluster.

[0149] The filter condition determination module 1003 is used to determine the selected filter condition for each subscribed message from the candidate filter conditions configured in the filter, based on the message type of the subscribed message.

[0150] The message filtering module 1004 is used to filter subscribed messages if they meet the selected filtering conditions.

[0151] In some embodiments, the number of selected filtering conditions is multiple, and the message filtering module 1004 is configured to: extract a target filtering condition from each selected filtering condition; perform filtering detection on the subscription message based on the target filtering condition, and determine the filtering detection result between the subscription message and the target filtering condition; if the filtering detection result indicates that the subscription message does not meet the target filtering condition, extract the target filtering condition from the remaining selected filtering conditions, and return to the step of performing filtering detection on the subscription message based on the target filtering condition; if the filtering detection result indicates that the subscription message meets the target filtering condition, filter the subscription message.

[0152] In some embodiments, the message filtering module is used to: obtain the expected detection duration of each selected filtering condition; and extract the target filtering condition with the shortest expected detection duration from the selected filtering conditions.

[0153] In some embodiments, the message filtering device 1000 further includes:

[0154] The instruction response module is used to respond to the condition addition instruction sent by the message cluster control platform, and to retrieve the new filtering condition and the condition configuration information of the new filtering condition from the cache database of the message cluster control platform.

[0155] The target filter determination module is used to determine the target filter for configuring new filtering conditions from among the filters based on the condition configuration information.

[0156] The condition configuration module is used to cache newly added filter conditions in the local cache area of ​​the target filter and configure the newly added filter conditions in the target filter.

[0157] In some embodiments, the condition configuration information includes the type of new filter condition. The message filtering device 1000 further includes:

[0158] The type matching module is used to match the type of the new filter condition with the filter condition type of each candidate filter condition in the target filter.

[0159] The same type of filter condition determination module is used to determine the same type of filter condition for the new filter condition from each candidate filter condition if there is a filter condition type that matches the new condition type.

[0160] The duplicate verification module is used to determine that no new filter condition should be configured if the newly added filter condition is successfully verified to be a duplicate of the same type of filter condition.

[0161] In some embodiments, the message filtering device 1000 further includes:

[0162] The failure criterion acquisition module is used to acquire the failure criteria for each candidate filtering condition in the filter.

[0163] The validity detection module is used to perform validity tests on each candidate filtering condition according to the failure criteria, and determine the validity detection result of the candidate filtering condition.

[0164] The failure condition removal module is used to remove candidate filtering conditions from the local cache area of ​​the filter when the validity test result indicates that the candidate filtering condition has failed.

[0165] Each module in the aforementioned message filtering device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0166] In some embodiments, a computer device is provided, which may be a terminal or server integrated with a consumer end, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as subscribed messages, candidate filtering conditions, and selected filtering conditions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a message filtering method.

[0167] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0169] In response to a message retrieval operation, determine the consumer group corresponding to the message retrieval operation;

[0170] The consumer group's matching filter is invoked to retrieve the subscribed messages from the message cluster; multiple filters are configured on the consumer side; subscribed messages are produced and published in the message cluster by the producer;

[0171] For each subscribed message, the selected filtering condition for the subscribed message is determined from the candidate filtering conditions configured in the filter, based on the message type of the subscribed message.

[0172] Subscribed messages are filtered if they meet the selected filtering criteria.

[0173] In some embodiments, when the processor executes a computer program, it further performs the following steps:

[0174] Extract the target filter condition from each selected filter condition;

[0175] The subscription messages are filtered and detected based on the target filtering conditions to determine the filtering detection results between the subscription messages and the target filtering conditions.

[0176] If the filtering detection result indicates that the subscribed message does not meet the target filtering condition, the target filtering condition is extracted from the remaining selected filtering conditions, and the step of filtering the subscribed message based on the target filtering condition is returned.

[0177] If the filtering detection results indicate that the subscribed messages meet the target filtering conditions, then the subscribed messages are filtered.

[0178] In some embodiments, when the processor executes a computer program, it further performs the following steps:

[0179] Obtain the expected detection time for each selected filter condition;

[0180] Select the target filter condition with the shortest expected detection time from the selected filter conditions.

[0181] In some embodiments, when the processor executes a computer program, it further performs the following steps:

[0182] In response to the condition addition command sent by the message cluster control platform, the new filter condition and the condition configuration information of the new filter condition are obtained from the cache database of the message cluster control platform.

[0183] Based on the condition configuration information, determine the target filter for configuring new filtering conditions from each filter;

[0184] The newly added filter conditions are cached in the local cache area of ​​the target filter, and the newly added filter conditions are configured in the target filter.

[0185] In some embodiments, when the processor executes a computer program, it further performs the following steps:

[0186] The type of the newly added filter condition is matched with the type of the filter condition of each candidate filter condition in the target filter.

[0187] If a filter condition type that matches the new condition type exists, then filter conditions of the same type as the new filter condition are determined from each candidate filter condition.

[0188] If the newly added filter condition is repeatedly validated against the same type of filter condition, and the validation is successful, then it is determined that no new filter condition will be configured.

[0189] In some embodiments, when the processor executes a computer program, it further performs the following steps:

[0190] Obtain the failure criteria for each candidate filter condition in the filter;

[0191] For each candidate filtering condition, the effectiveness of the candidate filtering condition is tested according to the failure criteria, and the effectiveness test result of the candidate filtering condition is determined.

[0192] If the validity test results indicate that the candidate filtering condition has become invalid, the candidate filtering condition is removed from the local cache area of ​​the filter.

[0193] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0194] In response to a message retrieval operation, determine the consumer group corresponding to the message retrieval operation;

[0195] The consumer group's matching filter is invoked to retrieve the subscribed messages from the message cluster; multiple filters are configured on the consumer side; subscribed messages are produced and published in the message cluster by the producer;

[0196] For each subscribed message, the selected filtering condition for the subscribed message is determined from the candidate filtering conditions configured in the filter, based on the message type of the subscribed message.

[0197] Subscribed messages are filtered if they meet the selected filtering criteria.

[0198] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0199] Extract the target filter condition from each selected filter condition;

[0200] The subscription messages are filtered and detected based on the target filtering conditions to determine the filtering detection results between the subscription messages and the target filtering conditions.

[0201] If the filtering detection result indicates that the subscribed message does not meet the target filtering condition, the target filtering condition is extracted from the remaining selected filtering conditions, and the step of filtering the subscribed message based on the target filtering condition is returned.

[0202] If the filtering detection results indicate that the subscribed messages meet the target filtering conditions, then the subscribed messages are filtered.

[0203] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0204] Obtain the expected detection time for each selected filter condition;

[0205] Select the target filter condition with the shortest expected detection time from the selected filter conditions.

[0206] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0207] In response to the condition addition command sent by the message cluster control platform, the new filter condition and the condition configuration information of the new filter condition are obtained from the cache database of the message cluster control platform.

[0208] Based on the condition configuration information, determine the target filter for configuring new filtering conditions from each filter;

[0209] The newly added filter conditions are cached in the local cache area of ​​the target filter, and the newly added filter conditions are configured in the target filter.

[0210] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0211] The type of the newly added filter condition is matched with the type of the filter condition of each candidate filter condition in the target filter.

[0212] If a filter condition type that matches the new condition type exists, then filter conditions of the same type as the new filter condition are determined from each candidate filter condition.

[0213] If the newly added filter condition is repeatedly validated against the same type of filter condition, and the validation is successful, then it is determined that no new filter condition will be configured.

[0214] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0215] Obtain the failure criteria for each candidate filter condition in the filter;

[0216] For each candidate filtering condition, the effectiveness of the candidate filtering condition is tested according to the failure criteria, and the effectiveness test result of the candidate filtering condition is determined.

[0217] If the validity test results indicate that the candidate filtering condition has become invalid, the candidate filtering condition is removed from the local cache area of ​​the filter.

[0218] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0219] In response to a message retrieval operation, determine the consumer group corresponding to the message retrieval operation;

[0220] The consumer group's matching filter is invoked to retrieve the subscribed messages from the message cluster; multiple filters are configured on the consumer side; subscribed messages are produced and published in the message cluster by the producer;

[0221] For each subscribed message, the selected filtering condition for the subscribed message is determined from the candidate filtering conditions configured in the filter, based on the message type of the subscribed message.

[0222] Subscribed messages are filtered if they meet the selected filtering criteria.

[0223] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0224] Extract the target filter condition from each selected filter condition;

[0225] The subscription messages are filtered and detected based on the target filtering conditions to determine the filtering detection results between the subscription messages and the target filtering conditions.

[0226] If the filtering detection result indicates that the subscribed message does not meet the target filtering condition, the target filtering condition is extracted from the remaining selected filtering conditions, and the step of filtering the subscribed message based on the target filtering condition is returned.

[0227] If the filtering detection results indicate that the subscribed messages meet the target filtering conditions, then the subscribed messages are filtered.

[0228] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0229] Obtain the expected detection time for each selected filter condition;

[0230] Select the target filter condition with the shortest expected detection time from the selected filter conditions.

[0231] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0232] In response to the condition addition command sent by the message cluster control platform, the new filter condition and the condition configuration information of the new filter condition are obtained from the cache database of the message cluster control platform.

[0233] Based on the condition configuration information, determine the target filter for configuring new filtering conditions from each filter;

[0234] The newly added filter conditions are cached in the local cache area of ​​the target filter, and the newly added filter conditions are configured in the target filter.

[0235] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0236] The type of the newly added filter condition is matched with the type of the filter condition of each candidate filter condition in the target filter.

[0237] If a filter condition type that matches the new condition type exists, then filter conditions of the same type as the new filter condition are determined from each candidate filter condition.

[0238] If the newly added filter condition is repeatedly validated against the same type of filter condition, and the validation is successful, then it is determined that no new filter condition will be configured.

[0239] In some embodiments, when a computer program is executed by a processor, it further performs the following steps:

[0240] Obtain the failure criteria for each candidate filter condition in the filter;

[0241] For each candidate filtering condition, the effectiveness of the candidate filtering condition is tested according to the failure criteria, and the effectiveness test result of the candidate filtering condition is determined.

[0242] If the validity test results indicate that the candidate filtering condition has become invalid, the candidate filtering condition is removed from the local cache area of ​​the filter.

[0243] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the acquisition, storage, processing, and transmission of the data all comply with relevant laws and regulations.

[0244] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0245] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0246] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A message filtering method, characterized by, When applied to the consumer end, the method includes: In response to a message retrieval operation, determine the consumer group corresponding to the message retrieval operation; The filter matched by the consumer group is invoked to retrieve the subscription messages subscribed to by the consumer group from the message cluster; the consumer is configured with multiple filters, each of which corresponds to a consumer group of the consumer; the subscription messages are produced by the producer and published in the message cluster. For each of the subscription messages, the selected filtering condition for the subscription message is determined from the candidate filtering conditions configured in the filter, based on the message type of the subscription message; the candidate filtering conditions are used to filter invalid messages for the consumer group. When there are multiple selected filtering conditions, the expected detection time of each selected filtering condition is obtained; the expected detection time of the selected filtering condition is the predicted detection time when the selected filtering condition is used to filter and detect any subscribed message. Select the target filter condition with the shortest expected detection time from the selected filter conditions; The subscription message is filtered and detected based on the target filtering conditions to determine the filtering detection result of the subscription message and the target filtering conditions; If the filtering detection result indicates that the subscription message meets the target filtering condition, the subscription message is filtered and discarded. In response to a condition addition instruction sent by the message cluster control platform, the system retrieves the new filtering condition and the condition configuration information of the new filtering condition from the cache database of the message cluster control platform. Based on the condition configuration information, a target filter for configuring the new filtering conditions is determined from each of the filters; The newly added filter condition is cached in the local cache area of ​​the target filter, and the newly added filter condition is configured in the target filter.

2. The method of claim 1, wherein, The method further includes: If the filtering detection result indicates that the subscription message does not meet the target filtering condition, the target filtering condition is extracted from the remaining selected filtering conditions, and the step of filtering the subscription message based on the target filtering condition is returned.

3. The method according to claim 1 or 2, characterized in that, The condition configuration information includes the type of new condition added to the new filter condition; the method further includes: The type of the newly added filter condition is matched with the type of the filter condition of each candidate filter condition in the target filter. If a filter condition type that matches the newly added condition type exists, then a filter condition of the same type as the newly added filter condition is determined from each of the candidate filter conditions. If the newly added filter condition is repeatedly verified against the existing filter conditions of the same type, and the verification is successful, it is determined that the newly added filter condition will not be configured.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain the failure criteria for each of the candidate filtering conditions in the filter; For each candidate filtering condition, an effectiveness test is performed on the candidate filtering condition according to the failure criteria to determine the effectiveness test result of the candidate filtering condition; If the validity detection result indicates that the candidate filtering condition has failed, the candidate filtering condition is removed from the local cache area of ​​the filter.

5. A message filtering device, characterized by The device includes: An operation response module is used to respond to a message extraction operation and determine the consumer group corresponding to the message extraction operation. The filter invocation module is used to invoke the filter matched by the consumer group to obtain each subscription message subscribed by the consumer group from the message cluster; the consumer is configured with multiple filters, each of which corresponds to a consumer group of the consumer; the subscription message is produced by the producer and published in the message cluster; The selected filter condition determination module is used to determine the selected filter condition for each subscription message from the candidate filter conditions configured in the filter, based on the message type of the subscription message; the candidate filter conditions are used to filter invalid messages for the consumer group. A message filtering module is configured to, when there are multiple selected filtering conditions, obtain the expected detection duration of each selected filtering condition; the expected detection duration of each selected filtering condition is the predicted detection duration when filtering any subscribed message using the selected filtering condition; extract the target filtering condition with the shortest expected detection duration from the selected filtering conditions; perform filtering detection on the subscribed message based on the target filtering condition to determine the filtering detection result of the subscribed message and the target filtering condition; if the filtering detection result indicates that the subscribed message meets the target filtering condition, filter the subscribed message and discard the subscribed message; The instruction response module is used to respond to the condition addition instruction sent by the message cluster control platform, and to obtain the new filtering condition and the condition configuration information of the new filtering condition from the cache database of the message cluster control platform. The target filter determination module is used to determine, based on the condition configuration information, a target filter for configuring the new filtering conditions from each of the filters; The condition configuration module is used to cache the newly added filter conditions in the local cache area of ​​the target filter and configure the newly added filter conditions in the target filter.

6. The apparatus of claim 5, wherein, The message filtering module is configured to: when the filtering detection result indicates that the subscription message does not meet the target filtering condition, extract the target filtering condition from the remaining selected filtering conditions, and return to the step of filtering the subscription message based on the target filtering condition.

7. The apparatus of claim 5 or 6, wherein, The condition configuration information includes the type of new condition added to the new filtering condition; the device further includes: The type matching module is used to match the type of the new condition of the new filter condition with the type of the filter condition of each candidate filter condition in the target filter. The same type of filtering condition determination module is used to determine the same type of filtering condition of the new filtering condition from each of the candidate filtering conditions if there is a filtering condition type that matches the new condition type. The duplicate verification module is used to determine that the new filter condition should not be configured if the new filter condition is successfully verified as a duplicate of the existing filter condition.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images