Data detection method and device, equipment, medium and program product

Abstract

The invention relates to the technical field of information detection, and discloses a data detection method and device, equipment, a medium and a program product.The method comprises the steps that a configuration page is displayed in response to a configuration instruction of a target service scene; in response to an interaction instruction for the configuration page, configuration information and a first detection rule in the target business scene are obtained, the first detection rule is obtained based on the detection field and a preset rule base, and the preset rule base is suitable for multiple business scenes; and displaying a target detection result of the detection data source, the target detection result being obtained by performing data detection on the detection data source based on a processing engine corresponding to the detection data source and a second detection rule, and the second detection rule being obtained by converting the first detection rule based on a rule conversion mode corresponding to the processing engine. According to the method and the device, the detection data source can be detected without writing detection logic when the method and the device are oriented to different business scenes, so that the data detection efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information detection technology, and in particular to a data detection method, device, equipment, medium and program product. Background Art

[0002] In some business scenarios, it is necessary to detect the data flow in the business scenario. The detection process depends on the detection logic and detection rules corresponding to the business scenario. Since different business scenarios require different detection logics, this will lead to low data detection efficiency in new business scenarios. Summary of the invention

[0003] In view of this, the present disclosure provides a data detection method, apparatus, device, medium and program product to solve the problem of low data detection efficiency.

[0004] In a first aspect, the present disclosure provides a data detection method, the method comprising:

[0005] In response to the configuration instructions of the target business scenario, display a configuration page;

[0006] In response to an interactive instruction for a configuration page, configuration information and a first detection rule in a target business scenario are obtained, where the configuration information includes a detection data source and a detection field in the detection data source, and the first detection rule is obtained based on the detection field and a preset rule base, where the preset rule base is applicable to a variety of business scenarios;

[0007] Display the target detection result of the detection data source. The target detection result is obtained by performing data detection on the detection data source based on the corresponding processing engine of the detection data source and the second detection rule. The second detection rule is obtained by converting the first detection rule based on the rule conversion method corresponding to the processing engine.

[0008] In a second aspect, the present disclosure provides a data detection method, the method comprising:

[0009] Obtain configuration information under the target business scenario, including the detection data source and the detection fields in the detection data source;

[0010] Obtaining a first detection rule in a target business scenario, where the first detection rule is obtained based on a detection field and a preset rule base, and the preset rule base is applicable to a variety of business scenarios;

[0011] Determine a corresponding processing engine based on the detection data source, and convert the first detection rule based on a rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine;

[0012] Based on the processing engine and the second detection rule, data detection is performed on the detection data source to obtain a target detection result.

[0013] In a third aspect, the present disclosure provides a data detection device, the device comprising:

[0014] A configuration page display module, used to display a configuration page in response to a configuration instruction of a target business scenario;

[0015] A configuration module, configured to obtain configuration information and a first detection rule in a target business scenario in response to an interactive instruction for a configuration page, wherein the configuration information includes a detection data source and a detection field in the detection data source, and the first detection rule is obtained based on the detection field and a preset rule base, and the preset rule base is applicable to a variety of business scenarios;

[0016] The detection result display module is used to display the target detection result of the detection data source. The target detection result is obtained by performing data detection on the detection data source based on the corresponding processing engine of the detection data source and the second detection rule. The second detection rule is obtained by converting the first detection rule based on the rule conversion method corresponding to the processing engine.

[0017] In a fourth aspect, the present disclosure provides a data detection device, the device comprising:

[0018] A configuration information acquisition module is used to acquire configuration information in a target business scenario, where the configuration information includes a detection data source and detection fields in the detection data source;

[0019] A rule acquisition module, used to acquire a first detection rule in a target business scenario, where the first detection rule is obtained based on a detection field and a preset rule base, and the preset rule base is applicable to a variety of business scenarios;

[0020] A conversion module, used to determine a corresponding processing engine based on a detection data source, and convert the first detection rule based on a rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine;

[0021] The detection module is used to perform data detection on the detection data source based on the processing engine and the second detection rule to obtain the target detection result.

[0022] In a fifth aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data monitoring method of the first aspect, the second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0023] In a sixth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data monitoring method of the first aspect, the second aspect or any corresponding embodiment thereof.

[0024] In a seventh aspect, the present disclosure provides a computer program product, including computer instructions, which are used to enable a computer to execute the data monitoring method of the first aspect, the second aspect or any corresponding embodiment thereof.

[0025] The data detection method provided by the embodiment of the present disclosure obtains configuration information and a first detection rule in a target business scenario in response to an interactive instruction for a configuration page, wherein the first detection rule is obtained based on a detection field in the configuration information and a preset rule library, and the preset rule library is applicable to a variety of business scenarios, so that the first detection rule can be configured for a variety of business scenarios; at the same time, the first detection rule is converted into a second detection rule based on a rule conversion method corresponding to a processing engine corresponding to a detection data source, and different processing engines corresponding to different business scenarios can all be converted to obtain corresponding second detection rules; and, data detection is performed on the detection data source through the second detection rule and the processing engine, so that when facing different business scenarios, the detection data source can be detected without writing detection logic, thereby improving the efficiency of data detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 is an application schematic diagram of the data detection method according to an embodiment of the present disclosure;

[0028] Figure 2 is a flow chart of a data detection method according to an embodiment of the present disclosure;

[0029] Figure 3 is a schematic diagram of a specific configuration page in the data detection method according to an embodiment of the present disclosure;

[0030] Figure 4 is a flow chart of a data detection method according to an embodiment of the present disclosure;

[0031] Figure 5 is a flow chart of another data detection method according to an embodiment of the present disclosure;

[0032] Figure 6 is a flow chart of another data detection method according to an embodiment of the present disclosure;

[0033] Figure 7 is a schematic diagram of obtaining a first detection rule in a data detection method according to an embodiment of the present disclosure;

[0034] Figure 8 is a flow chart of a data detection method according to an embodiment of the present disclosure;

[0035] Fig. 9 is a schematic diagram of obtaining a first query result by matching in a data detection method according to an embodiment of the present disclosure;

[0036] Fig.10 is a schematic diagram of obtaining a second query result by matching in the data detection method according to an embodiment of the present disclosure;

[0037] Fig.11 is a schematic diagram of obtaining a target detection result in a data detection method according to an embodiment of the present disclosure;

[0038] Fig.12 is a schematic diagram of a specific application of the data detection method according to an embodiment of the present disclosure;

[0039] Fig.13 is a structural block diagram of a data detection device according to an embodiment of the present disclosure;

[0040] Fig.14 is a structural block diagram of a data detection device according to an embodiment of the present disclosure;

[0041] Fig.15 It is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0043] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0044] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0045] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0046] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0047] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0048] In related technologies, in order to achieve data detection in different business scenarios, the following two solutions are usually adopted. The first solution is to develop a data processing task for each real-time data stream; the second solution is to write the data stream into the database, and convert the data detection rules into SQL (Structured Query Language) through a scheduled task (cronjob task) to query the database. Both of the above solutions require customized development for business scenarios and writing different detection logic, which leads to the problem of low data detection efficiency in new business scenarios.

[0049] Based on this, the present disclosure provides a data detection method, which responds to interactive instructions for a configuration page, obtains configuration information and a first detection rule under a target business scenario, wherein the first detection rule is obtained based on a detection field in the configuration information and a preset rule library, and the preset rule library is applicable to a variety of business scenarios, so that the first detection rule can be configured for a variety of business scenarios; at the same time, based on a rule conversion method corresponding to a processing engine corresponding to a detection data source, the first detection rule is converted to obtain a second detection rule, and corresponding second detection rules can be converted for different processing engines; and, data detection is performed on the detection data source through the second detection rule and the processing engine, so that when facing different business scenarios and different processing engines, there is no need to write detection logic, and the detection data source can be detected, thereby improving the efficiency of data detection.

[0050] Figure 1 is an application diagram of the data detection method according to an embodiment of the present disclosure, please refer to Figure 1 The data detection method of the embodiment of the present disclosure provides an interactive window to the user in the form of a configuration page, and the configuration page can be deployed in multiple devices, such as mobile terminals such as mobile phones, computers and tablets. The user can configure the configuration information and detection rules related to data detection on the configuration page. At the same time, the data detection method of the embodiment of the present disclosure can be deployed in another terminal or server. After the user configures the configuration information and detection rules, the configuration information and detection rules are obtained, and the corresponding detection data source is detected to obtain the target detection result.

[0051] According to an embodiment of the present disclosure, an embodiment of a data detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0052] On the one hand, in this embodiment, a data detection method is provided, which can be used in a terminal or a server. Figure 2 is a flow chart of a data detection method according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:

[0053] Step S201, in response to a configuration instruction of a target business scenario, a configuration page is displayed.

[0054] In the present disclosure, Figure 1 As shown, if the user triggers a configuration instruction for the target business scenario, a response is generated to the configuration instruction, and a configuration page is displayed in the interface. The configuration page provides the user with a configuration window for configuring the detection data source and the detection field. Among them, the detection data source defines the data source applicable to this configuration, and the combination of detection fields is used to characterize the detection rules corresponding to this configuration. The user can add or delete various types of configuration windows through interactive operations on the configuration page, and enter detection fields in the configuration window through interactive operations on the configuration window.

[0055] In an optional embodiment, multiple types of configuration windows can be set in the configuration page. Different types of configuration windows have different combination relationships with other configuration windows. By adjusting the type of configuration window and the position of different types of configuration windows, the selection and configuration of different detection rules can be achieved.

[0056] For example, Figure 3is a schematic diagram of a specific configuration page in the data detection method according to an embodiment of the present disclosure, such as Figure 3 As shown, the configuration page includes 5 configuration windows, among which, from top to bottom, the first configuration window to the third configuration window are all configuration windows of the AND type, indicating that the corresponding detection fields are in an AND combination relationship with the detection fields corresponding to the previous configuration window, and the fourth configuration window and the fifth configuration window are all configuration windows of the OR type, indicating that the corresponding detection fields are in an OR combination relationship with the detection fields corresponding to the previous configuration window. Then, Figure 2 The detection field corresponding to the configuration page includes 5 detection fields, among which detection field a, detection field b and detection field c are in an and logical relationship, and detection field c, detection field d and detection field e are in an or logical relationship.

[0057] Step S202: In response to an interactive instruction for a configuration page, obtain configuration information and a first detection rule in a target business scenario.

[0058] In the disclosed embodiment, the configuration information includes a detection data source and a detection field in the detection data source. The detection data source and the detection field are input by the user through interaction in the configuration window of the configuration page. When the user completes the input of the configuration information and triggers the interactive instruction for the configuration page, the configuration information interactively input by the user in the configuration page is obtained, thereby obtaining the detection data source and the detection field for the target business scenario.

[0059] In the disclosed embodiment, the first detection rule is obtained based on the detection field and the preset rule base, and the preset rule base is applicable to a variety of business scenarios, that is, the preset rule base is a general rule base, and the rules and rule encoding methods contained therein are generally applicable to a variety of business scenarios. Although the detection field represents the user's configuration of the detection rule, it is still a data fragment. The rule code or script corresponding to the detection field needs to be used in the data detection process. Therefore, the first detection rule is generated based on the detection field and the preset rule base.

[0060] In an optional implementation, if different business scenarios use different rule encoding methods, the preset rule library can preset corresponding rule encoding methods for various business scenarios. When generating the first detection rule, according to the target business scenario, the rule encoding method corresponding to the target business scenario is obtained from the preset rule library, and the first detection rule is generated in combination with the detection field.

[0061] In an optional implementation, if different business scenarios use different rule encoding methods, when a new business scenario is added, the rule encoding method corresponding to the new business scenario is synchronously added to the preset rule library, so that the first detection rule can be generated for all business scenarios to ensure the normal progress of data detection.

[0062] Step S203, displaying the target detection result of the detection data source.

[0063] In the disclosed embodiment, the target detection result is obtained by performing data detection on the detection data source based on the corresponding processing engine of the detection data source and the second detection rule. Among them, the second detection rule is obtained by converting the first detection rule based on the rule conversion method corresponding to the processing engine; because different data sources will use different processing engines, it is necessary to convert the first detection rule into a second detection rule that can be recognized and used by the processing engine corresponding to the detection data source. Based on the processing engine corresponding to the detection data source and the second detection rule, data matching and detection are performed on the data stream of the detection data source to obtain the target detection result, and the target detection result is displayed in the interface, thereby showing the user the result of data detection on the detection data source.

[0064] In an optional implementation, in order to ensure that the target detection result can meet the user's detection requirements, after the first detection rule is converted to obtain the second detection rule, the detection data source can be initially detected and previewed to confirm whether the initial detection result obtained meets the preset requirements. Specifically, the data detection method of the embodiment of the present disclosure may also include the following steps:

[0065] Step a1, obtaining the initial detection result of the detection data source.

[0066] In step a1, the initial detection result is obtained by performing data detection on the detection data source based on the corresponding processing engine of the detection data source and the second detection rule; that is, after the first detection rule is converted to obtain the second detection rule, the detection data source is firstly detected once to obtain the initial detection result.

[0067] Step a2: display the initial detection result on the preview page.

[0068] In step a2, the initial detection result is displayed on the preview page to show the user the detection result that can be obtained by performing data detection on the detection data source according to the currently configured detection field. The user can confirm whether the initial detection result meets the preset requirements based on the content displayed on the preview page. If the initial detection result meets the preset requirements, it indicates that the current second detection rule can be used for subsequent data detection of the detection data source, and the second detection rule does not need to be modified; if the initial detection result does not meet the preset requirements, it indicates that the current second detection rule cannot be used for subsequent data detection of the detection data source, and the second detection rule needs to be modified.

[0069] Step a3: if the initial detection result does not meet the preset requirement, in response to the modification instruction for the first detection rule, a modified first detection rule is obtained to obtain a modified second detection rule.

[0070] In step a3, if the initial detection result does not meet the preset requirements, the user can modify the detection field in the configuration page to modify the first detection rule. When the user completes the modification of the detection field and triggers the modification instruction for the first detection rule, a response is generated to the modification instruction for the first detection rule, the modified first detection rule is obtained, and the modified first detection rule is converted to obtain the modified second detection rule. Based on the processing engine and the modified second detection rule, data detection is performed on the detection data source to obtain the target detection result.

[0071] In an optional implementation, if the detection result still cannot meet the preset requirements after the first detection rule is modified in step a3, the above steps a1 to a3 can be repeated to modify the first detection rule multiple times until the detection result can meet the preset requirements. Thus, after the second detection rule is converted, the detection data source is tested to obtain and display the initial detection result, so as to provide a window for confirming whether the detection rule meets the detection requirements. By continuously modifying the first detection rule, it is ensured that the target detection result finally obtained can meet the detection requirements.

[0072] In an optional implementation, the configuration information also includes a processing method for the target detection result. After the target detection result is obtained by performing data detection on the detection data source, the target detection result can also be pushed accordingly according to the processing method for the target detection result. Specifically, if the processing method indicates that the target detection result needs to be pushed to the target object, the display of the conversation page with the target object in the instant messaging application is triggered; the target detection result is displayed in the conversation page and sent to the target object. Among them, the specific triggered instant messaging application can be limited in the processing method for the target detection result, and the default instant messaging application can be set to trigger the default instant messaging application when the triggered instant messaging application is not limited in the processing method for the target detection result.

[0073] In an optional implementation, there may also be a processing method that characterizes remote calls and a processing method that characterizes other business triggering logics; the processing method of the target detection result may include more or fewer types as described above, and no specific limitation is made here.

[0074] The data detection method provided by the embodiment of the present disclosure obtains configuration information and a first detection rule in a target business scenario in response to an interactive instruction for a configuration page, wherein the first detection rule is obtained based on a detection field in the configuration information and a preset rule library, and the preset rule library is applicable to a variety of business scenarios, so that the first detection rule can be configured for a variety of business scenarios; at the same time, the first detection rule is converted into a second detection rule based on a rule conversion method corresponding to a processing engine corresponding to a detection data source, and different processing engines corresponding to different business scenarios can all be converted to obtain corresponding second detection rules; and, data detection is performed on the detection data source through the second detection rule and the processing engine, so that when facing different business scenarios, the detection data source can be detected without writing detection logic, thereby improving the efficiency of data detection.

[0075] On the other hand, in this embodiment, a data detection method is provided, which can be used in a terminal or a server. Figure 4 is a flow chart of a data detection method according to an embodiment of the present disclosure, such as Figure 4 As shown, the process includes the following steps:

[0076] Step S401, obtaining configuration information in a target business scenario.

[0077] In the embodiment of the present disclosure, the configuration information includes the detection data source and the detection field in the detection data source. As described in step S201 and step S202 above, the configuration information is obtained from the configuration page after responding to the interactive instruction for the configuration page. For details, please refer to Figure 2 Step S201 and step S202 of the illustrated embodiment are not described in detail here.

[0078] Step S402: Obtain a first detection rule in a target business scenario.

[0079] In the embodiment of the present disclosure, the first detection rule is obtained based on the detection field and the preset rule library, and the preset rule library is applicable to various business scenarios; for details on obtaining the first detection rule, please refer to Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0080] Step S403, determining the corresponding processing engine based on the detection data source, and converting the first detection rule based on the rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine.

[0081] In the disclosed embodiment, the data processing architecture or system used by the detection data source is determined to obtain the processing engine corresponding to the data source. Based on the rule conversion method corresponding to the processing engine, the first detection rule is converted into a Boolean expression recognizable by the processing engine to obtain the second detection rule.

[0082] Step S404: Based on the processing engine and the second detection rule, data detection is performed on the detection data source to obtain a target detection result.

[0083] In the disclosed embodiment, based on the processing engine and the second detection rule, data matching and detection are performed on the data stream in the detection data source to obtain a target detection result.

[0084] The data detection method provided by the embodiment of the present disclosure obtains configuration information and a first detection rule in a target business scenario in response to an interactive instruction for a configuration page, wherein the first detection rule is obtained based on a detection field in the configuration information and a preset rule library, and the preset rule library is applicable to a variety of business scenarios, so that the first detection rule can be configured for a variety of business scenarios; at the same time, the first detection rule is converted into a second detection rule based on a rule conversion method corresponding to a processing engine corresponding to a detection data source, and different processing engines corresponding to different business scenarios can all be converted to obtain corresponding second detection rules; and, data detection is performed on the detection data source through the second detection rule and the processing engine, so that when facing different business scenarios, the detection data source can be detected without writing detection logic, thereby improving the efficiency of data detection.

[0085] In this embodiment, a data detection method is provided, which can be used in a terminal or a server. Figure 5 is a flow chart of another data detection method according to an embodiment of the present disclosure. Figure 5 As shown, the process includes the following steps:

[0086] Step S501: Obtain configuration information for the target business scenario. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0087] Step S502: Obtain the first detection rule in the target business scenario. Figure 4 Step S402 of the illustrated embodiment will not be described in detail here.

[0088] Step S503, determining the corresponding processing engine based on the detection data source, and converting the first detection rule based on the rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine.

[0089] Specifically, step S503 includes:

[0090] Step S5031, obtaining a rule conversion script corresponding to the processing engine.

[0091] In the disclosed embodiment, in order to ensure that the second detection rule obtained by converting the first detection rule can be recognized by the processing engine, it is necessary to obtain the rule conversion script corresponding to the processing engine to convert the first detection rule. Among them, the rule conversion script is used to characterize the execution logic of the rule conversion method; for each processing engine, there is a corresponding rule conversion script. For example, for the flink processing engine, the corresponding rule conversion script is a user-defined function (), and for the doris and clickhouse processing engines, the corresponding rule conversion script is a sql (Structured Query Language) query template.

[0092] In an optional implementation, if a new processing engine is added, a rule conversion script corresponding to the new processing engine is added simultaneously, so that the first detection rule can be converted for all processing engines to ensure normal data detection.

[0093] Step S5032: convert the first detection rule based on the rule conversion script to obtain a second detection rule corresponding to the processing engine.

[0094] In the embodiment of the present disclosure, the first detection rule is processed based on the rule conversion script, and the first detection rule is converted into a Boolean expression that can be recognized by the processing engine to obtain the second detection rule.

[0095] Step S504: Based on the processing engine and the second detection rule, data detection is performed on the detection data source to obtain a target detection result. Figure 4 Step S404 of the illustrated embodiment will not be described in detail here.

[0096] The data detection method provided by the embodiment of the present disclosure is provided with a corresponding rule conversion script for each processing engine, so as to convert a first detection rule into a second detection rule that can be supported by the processing engine, so that data detection can be performed on the detection data source for different processing engines.

[0097] In this embodiment, a data detection method is provided, which can be used in a terminal or a server. Figure 6 is a flow chart of another data detection method according to an embodiment of the present disclosure, such as Figure 6 As shown, the process includes the following steps:

[0098] Step S601, obtain the configuration information of the target business scenario. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0099] Step S602: Obtain the first detection rule in the target business scenario. Figure 4Step S402 of the illustrated embodiment will not be described in detail here.

[0100] Step S603, determining the corresponding processing engine based on the detection data source, and converting the first detection rule based on the rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine.

[0101] Specifically, Figure 7 is a schematic diagram of obtaining a first detection rule in a data detection method according to an embodiment of the present disclosure, such as Figure 7 As shown, step S603 includes:

[0102] Step S6031, converting the first detection rule based on the rule conversion script to obtain a third detection rule.

[0103] In the present disclosure, Figure 7 As shown, the detection rule obtained by converting the first detection rule is the third detection rule; the third detection rule is a detection rule obtained by a rule configuration, specifically, a detection rule applicable to the corresponding processing engine obtained according to the configuration information obtained after each response to the interactive instruction for the configuration page. The second detection rule is an aggregation of all third detection rules applicable to the same business entity in the target business scenario. The second detection rule includes not only the detection rule converted from the first detection rule corresponding to the current configuration information, but also the detection rule applicable to the business entity that has been converted at a historical moment.

[0104] In the present disclosure, Figure 7 As shown, refer to the above Figure 5 In step S503 of the illustrated embodiment, based on a rule conversion script corresponding to a processing engine corresponding to a processing detection data source, the first detection rule is converted to obtain a third detection rule.

[0105] Step S6032: Obtain the business entity to which the first detection rule applies.

[0106] In the embodiment of the present disclosure, the business entity to which the first detection rule applies may be determined according to the business entity to which the detection rule applies included in the configuration information.

[0107] In an optional implementation, a first detection rule may be applicable to multiple business entities, that is, the detection rule in the configuration information may be applicable to multiple business entities; if the business entities to which the detection rule applies are not configured in the configuration information, it can be assumed that the first detection rule is applicable to all business entities.

[0108] In an optional implementation, the business entity to which the first detection rule applies can be represented by the business entity identifier; correspondingly, the business entity used by the first initial detection rule can be represented by configuring the business entity identifier in the configuration information. If the configuration information does not contain the business entity identifier, it can be assumed that the first detection rule applies to all business entities.

[0109] Step S6033: Based on the business entity to which the first detection rule is applicable, the third detection rules corresponding to the same business entity are aggregated to obtain the second detection rule under the target business scenario.

[0110] In the disclosed embodiment, all third detection rules applicable to the same business entity are aggregated into one data stream to obtain a second detection rule applicable to the business entity. For each business entity in the target business scenario, all third detection rules applicable thereto are aggregated respectively to obtain the second detection rule corresponding to each business entity respectively, and all second detection rules in the target business scenario are obtained. Thus, when performing data detection on the data stream corresponding to a certain business entity, the second detection rule corresponding to the business entity can be directly matched from the second detection rule in the target business scenario, which can reduce repeated parsing and conversion of detection rules, thereby improving the efficiency of data detection.

[0111] In an optional embodiment, if Figure 7 As shown, after the second detection rule is obtained through aggregation, the second detection rule of the target business scenario can also be persistently stored, and the latest detection rule of the same business entity can be cached. On the one hand, the latest detection rule of the same business entity is cached in the memory space, so that when performing data detection on the data stream corresponding to the business entity, the corresponding detection rule can be directly called from the memory space without the need to repeatedly parse and convert the rule, thereby improving the efficiency of data detection; on the other hand, the second detection rule of the target business scenario is persistently stored, and when the latest detection rule in the memory space becomes invalid due to expiration, restart, etc., the part of the detection rule can be taken out from the persistently stored second detection rule, thereby further improving the efficiency of data detection.

[0112] Step S604: Based on the processing engine and the second detection rule, data detection is performed on the detection data source to obtain the target detection result. Figure 4 Step S404 of the illustrated embodiment will not be described in detail here.

[0113] In this embodiment, a data detection method is provided, which can be used in a terminal or a server. Figure 8 is a flow chart of a data detection method according to an embodiment of the present disclosure, such as Figure 8 As shown, the process includes the following steps:

[0114] Step S801, obtain the configuration information of the target business scenario. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0115] Step S802: Obtain the first detection rule in the target business scenario. Figure 4 Step S402 of the illustrated embodiment will not be described in detail here.

[0116] Step S803: Determine the corresponding processing engine based on the detection data source, and convert the first detection rule based on the rule conversion method corresponding to the processing engine to obtain the second detection rule corresponding to the processing engine. Figure 4 Step S403 of the illustrated embodiment will not be described in detail here.

[0117] Step S804: Based on the processing engine and the second detection rule, data detection is performed on the detection data source to obtain a target detection result.

[0118] Specifically, step S804 includes:

[0119] Step S8041, obtaining the data stream of the detection data source.

[0120] In the embodiment of the present disclosure, the data stream of the detection data source is acquired in real time, and the acquired data stream is used as the target of data detection.

[0121] Step S8042: Based on the business entity in the data flow, a match is performed in the second detection rule to obtain a target detection rule.

[0122] In the embodiment of the present disclosure, based on the business entity in the data stream, matching is performed in the second detection rule, and the detection rule applicable to the business entity in the data stream is matched, thereby obtaining the target detection rule.

[0123] In an optional implementation, according to the different scopes of business entities to which the second detection rule applies, the target detection rule is obtained by matching in the second detection rule, which can be divided into the following two cases; that is, the target detection rule includes the first query result and the second query result, and the first query result and the second query result are respectively matched, which are as follows:

[0124] In the first case, Fig. 9 is a schematic diagram of obtaining a first query result by matching in a data detection method according to an embodiment of the present disclosure, such as Fig. 9As shown, based on the identifier of the business entity in the data stream, a query is performed in the second detection rule to obtain a first query result. Specifically, the identifier of the business entity in the data stream is queried in the second detection rule obtained by aggregation, and the second detection rule containing the identifier of the business entity in the data stream, that is, the second detection rule applicable to the business entity in the data stream, is determined as the first query result. Among them, if the latest detection rule is cached in the memory space, the query is performed in the latest detection rule, and if there is no latest cached rule in the memory space, the second detection rule stored persistently is called and queried therein.

[0125] The second case, Fig.10 is a schematic diagram of obtaining a second query result by matching in the data detection method according to an embodiment of the present disclosure, such as Fig.10 As shown, the second query result applicable to all business entities in the second detection rule is determined. Specifically, the detection rule that does not specify the identifier of the business entity in the second detection rule is determined as the second query result.

[0126] The first query result and the second query result obtained by matching are combined to obtain a target detection rule. Thus, the first query result applicable to the business entity of the data stream of the detection data source only and the second query result applicable to all business entities are determined as the target detection rule, thereby querying to obtain the detection rules of all business entities applicable to the detection data source, and performing comprehensive data detection on the data stream of the detection data source.

[0127] Step S8043: Detect the data stream based on the processing engine and the target detection rule to obtain the target detection result.

[0128] In the present disclosure, Fig.11 is a schematic diagram of obtaining a target detection result in a data detection method according to an embodiment of the present disclosure, such as Fig.11 As shown, based on the target detection rules and processing engine, data matching is performed on the data stream of the detection data source, and the data stream that successfully matches the target detection rules is obtained. The obtained data stream is post-processed and deduplicated to achieve result deduplication, and finally the target detection result is obtained.

[0129] The data detection method provided by the embodiment of the present disclosure can directly match the corresponding detection rule from the second detection rule corresponding to the same business entity when detecting the data flow corresponding to the same business entity, which can reduce repeated parsing and conversion of the detection rules, thereby improving the efficiency of data detection.

[0130] As a specific application example of the embodiment of the present disclosure, Fig.12 is a schematic diagram of a specific application of the data detection method according to an embodiment of the present disclosure, such as Fig.12As shown, the client creates data sources based on different processing engines, such as creating a bmq data stream type data source based on the flink processing engine, creating a clickhouse table type data source based on the clickhouse processing engine, and creating a doris table type data source based on the doris processing engine. First, based on the data source created by the client, enter the configuration information in the configuration page to configure the detection data source and detection field, and combine with the preset rule base to obtain the first detection rule; wherein the detection field can include table fields, operation symbols and values, etc.

[0131] Secondly, based on the rule conversion method for different processing engines configured in the rule conversion engine, the first detection rule is converted into a second detection rule that can be recognized by the processing engine corresponding to the detection data source. For example, the first detection rule corresponding to the bmq data stream type data source is converted into the second detection rule that can be recognized by the flink processing engine, the first detection rule corresponding to the clickhouse table type data source is converted into the second detection rule that can be recognized by the clickhouse processing engine, and the first detection rule corresponding to the doris table type data source is converted into the second detection rule that can be recognized by the doris processing engine.

[0132] Then, a preliminary test and rule preview are performed based on the second detection rule to determine whether the current second detection rule meets the detection requirements. If it does not meet the detection requirements, the first detection rule is modified by modifying the detection field in the configuration page, and then the second detection rule is modified.

[0133] Finally, the execution engine performs data detection on the data stream of the detection data source based on the second detection rule, and deduplicates the data detection result through the deduplication engine to obtain the final target detection result. There are multiple processing methods for the final target detection result. According to the processing method of the target detection result configured in the configuration information, the target detection result is subjected to corresponding processing operations, which can be specifically divided into two processing operations: submitting to the user for processing and submitting to the platform for processing; among them, submitting to the platform for processing can be divided into processing operations such as remote call, instant messaging application display, and other business trigger logic.

[0134] In the present embodiment, a data detection device is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the description has been made no further. As used below, the term "module" can implement the combination of software and / or hardware of the predetermined function. Although the device described in the following embodiments is preferably implemented with software, the implementation of hardware, or the combination of software and hardware is also possible and conceived.

[0135] This embodiment provides a data detection device, such as Fig.13As shown, including:

[0136] A configuration page display module 1301 is used to display a configuration page in response to a configuration instruction of a target business scenario;

[0137] Configuration module 1302, configured to obtain configuration information and a first detection rule in a target business scenario in response to an interactive instruction for a configuration page, wherein the configuration information includes a detection data source and a detection field in the detection data source, and the first detection rule is obtained based on the detection field and a preset rule base, and the preset rule base is applicable to multiple business scenarios;

[0138] The detection result display module 1303 is used to display the target detection result of the detection data source. The target detection result is obtained by performing data detection on the detection data source based on the corresponding processing engine of the detection data source and the second detection rule. The second detection rule is obtained by converting the first detection rule based on the rule conversion method corresponding to the processing engine.

[0139] In an optional embodiment, the device further comprises:

[0140] An initial detection result acquisition module is used to obtain an initial detection result of a detection data source, where the initial detection result is obtained by performing data detection on the detection data source based on a processing engine corresponding to the detection data source and a second detection rule;

[0141] A test result preview module, used to display the initial test results on a preview page;

[0142] The detection rule modification module is used to obtain a modified first detection rule in response to a modification instruction for the first detection rule if the initial detection result does not meet the preset requirements, so as to obtain a modified second detection rule. The target detection result is obtained based on the processing engine and the modified second detection rule.

[0143] In an optional implementation, the configuration information also includes a method for processing the target detection result, and the device further includes:

[0144] A conversation page display module, used to trigger the display of a conversation page with the target object in the instant messaging application if the processing method indicates that the target detection result needs to be pushed to the target object;

[0145] The detection result push module is used to display the target detection results in the session page and send them to the target object.

[0146] This embodiment also provides a data detection device, such as Fig.14 As shown, including:

[0147] The configuration information acquisition module 1401 is used to acquire configuration information in a target business scenario, where the configuration information includes a detection data source and a detection field in the detection data source;

[0148] A rule acquisition module 1402 is used to acquire a first detection rule in a target business scenario, where the first detection rule is obtained based on a detection field and a preset rule base, and the preset rule base is applicable to a variety of business scenarios;

[0149] The conversion module 1403 is used to determine the corresponding processing engine based on the detection data source, and convert the first detection rule based on the rule conversion method corresponding to the processing engine to obtain the second detection rule corresponding to the processing engine;

[0150] The detection module 1404 is used to perform data detection on the detection data source based on the processing engine and the second detection rule to obtain a target detection result.

[0151] In an optional implementation, the conversion module 1403 includes:

[0152] A conversion script acquisition unit, used to acquire a rule conversion script corresponding to the processing engine, the rule conversion script being used to characterize the execution logic of the rule conversion method;

[0153] The detection rule conversion unit is used to convert the first detection rule based on the rule conversion script to obtain a second detection rule corresponding to the processing engine.

[0154] In an optional implementation, the conversion module 1403 includes:

[0155] A rule conversion unit, configured to convert the first detection rule based on a rule conversion script to obtain a third detection rule;

[0156] An applicable entity acquisition unit, used to acquire a business entity to which the first detection rule is applicable;

[0157] The rule aggregation unit is used to aggregate the third detection rules corresponding to the same business entity based on the business entity to which the first detection rule is applicable, so as to obtain the second detection rule under the target business scenario.

[0158] In an optional embodiment, the device further comprises:

[0159] The rule storage module is used to persistently store the second detection rule of the target business scenario and cache the latest detection rule of the same business entity.

[0160] In an optional implementation, the detection module 1404 includes:

[0161] A data stream acquisition unit, used to acquire a data stream of a detection data source;

[0162] A rule matching unit, configured to match a service entity in a data stream in a second detection rule to obtain a target detection rule;

[0163] The data detection unit is used to detect the data stream based on the processing engine and the target detection rule to obtain the target detection result.

[0164] In an optional implementation, the rule matching unit includes:

[0165] A first query subunit, configured to query in a second detection rule based on an identifier of a service entity in a data stream to obtain a first query result;

[0166] The second query subunit is used to determine the second query result applicable to all business entities in the second detection rule to obtain the target detection rule, and the target detection rule includes the first query result and the second query result.

[0167] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0168] The data detection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0169] The present disclosure also provides an electronic device having the above Fig.13 and Fig.14 The data detection device shown.

[0170] See also Fig.15 , Fig.15 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure, such as Fig.15As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.15 A processor 10 is taken as an example.

[0171] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0172] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0173] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0174] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0175] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0176] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0177] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0178] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the present disclosure.

Claims

1. A data detection method, characterized in that: The method comprises: In response to the configuration instructions of the target business scenario, display a configuration page; In response to an interactive instruction for the configuration page, obtaining configuration information and a first detection rule in the target business scenario, the configuration information including a detection data source and a detection field in the detection data source, the first detection rule being obtained based on the detection field and a preset rule base, and the preset rule base being applicable to a variety of business scenarios; The target detection result of the detection data source is displayed, wherein the target detection result is obtained by performing data detection on the detection data source based on the processing engine corresponding to the detection data source and the second detection rule, and the second detection rule is obtained by converting the first detection rule based on the rule conversion method corresponding to the processing engine.

2. The method according to claim 1, characterized in that The method further comprises: Acquire an initial detection result of the detection data source, where the initial detection result is obtained by performing data detection on the detection data source based on a processing engine corresponding to the detection data source and the second detection rule; Displaying the initial detection result on a preview page; If the initial detection result does not meet the preset requirements, a modified first detection rule is obtained in response to a modification instruction for the first detection rule to obtain a modified second detection rule, and the target detection result is obtained based on the processing engine and the modified second detection rule.

3. The method according to claim 1, characterized in that The configuration information also includes a processing method for the target detection result, and the method further includes: If the processing method indicates that the target detection result needs to be pushed to the target object, then triggering the display of a conversation page with the target object in the instant messaging application; The target detection result is displayed in the session page and sent to the target object.

4. A data detection method, characterized in that: The method comprises: Acquire configuration information in a target business scenario, the configuration information including a detection data source and a detection field in the detection data source; Acquire a first detection rule in the target business scenario, where the first detection rule is obtained based on the detection field and a preset rule base, and the preset rule base is applicable to multiple business scenarios; Determine a corresponding processing engine based on the detection data source, and convert the first detection rule based on a rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine; Based on the processing engine and the second detection rule, data detection is performed on the detection data source to obtain a target detection result.

5. The method according to claim 4, characterized in that The converting the first detection rule based on the rule conversion mode corresponding to the processing engine to obtain the second detection rule corresponding to the processing engine includes: Acquire a rule conversion script corresponding to the processing engine, wherein the rule conversion script is used to represent the execution logic of the rule conversion method; The first detection rule is converted based on the rule conversion script to obtain a second detection rule corresponding to the processing engine.

6. The method according to claim 5, characterized in that The converting the first detection rule based on the rule conversion script to obtain a second detection rule corresponding to the processing engine includes: Convert the first detection rule based on the rule conversion script to obtain a third detection rule; Obtaining a business entity to which the first detection rule applies; Based on the business entity to which the first detection rule is applicable, third detection rules corresponding to the same business entity are aggregated to obtain a second detection rule under the target business scenario.

7. The method according to claim 6, characterized in that The method further comprises: The second detection rule of the target business scenario is persistently stored, and the latest detection rule of the same business entity is cached.

8. The method according to claim 4, characterized in that The performing data detection on the detection data source based on the processing engine and the second detection rule to obtain a target detection result includes: Obtaining a data stream from the detection data source; Based on the business entity in the data flow, matching is performed in the second detection rule to obtain a target detection rule; The data stream is detected based on the processing engine and the target detection rule to obtain the target detection result.

9. The method according to claim 8, characterized in that The matching in the second detection rule based on the service entity in the data flow to obtain the target detection rule includes: Based on the identifier of the service entity in the data flow, query in the second detection rule to obtain a first query result; A second query result applicable to all business entities in the second detection rule is determined to obtain the target detection rule, where the target detection rule includes the first query result and the second query result.

10. A data detection device, characterized in that: The device comprises: A configuration page display module, used to display a configuration page in response to a configuration instruction of a target business scenario; a configuration module, configured to obtain, in response to an interactive instruction for the configuration page, configuration information and a first detection rule in the target business scenario, wherein the configuration information includes a detection data source and a detection field in the detection data source, and the first detection rule is obtained based on the detection field and a preset rule base, and the preset rule base is applicable to multiple business scenarios; A detection result display module is used to display the target detection result of the detection data source, wherein the target detection result is obtained by performing data detection on the detection data source based on the corresponding processing engine of the detection data source and a second detection rule, and the second detection rule is obtained by converting the first detection rule based on a rule conversion method corresponding to the processing engine.

11. A data detection device, characterized in that: The device comprises: A configuration information acquisition module, used to acquire configuration information in a target business scenario, wherein the configuration information includes a detection data source and a detection field in the detection data source; A rule acquisition module, used to acquire a first detection rule in the target business scenario, where the first detection rule is obtained based on the detection field and a preset rule base, and the preset rule base is applicable to multiple business scenarios; a conversion module, configured to determine a corresponding processing engine based on the detection data source, and convert the first detection rule based on a rule conversion method corresponding to the processing engine to obtain a second detection rule corresponding to the processing engine; A detection module is used to perform data detection on the detection data source based on the processing engine and the second detection rule to obtain a target detection result.

12. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data detection method according to any one of claims 1 to 9 by executing the computer instructions.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data detection method according to any one of claims 1 to 9.

14. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the data detection method according to any one of claims 1 to 9.

Images