Line screening method and device, storage medium and electronic equipment

Abstract

Embodiments of the present application disclose a line screening method and device, a storage medium and an electronic device. The method comprises: configuring a first rule text, and performing orthogonal checking on the first rule text and all pre-stored second rule texts; when the checking result indicates that there is no intersection between the first rule text and the second rule texts, storing the first rule text; and screening a target line meeting a condition from a line database based on the first rule text. The present scheme can improve data query accuracy. The present scheme determines whether the rule texts are orthogonal before query, avoids the possibility that there is an orthogonal relationship between line sets, and improves line query accuracy without comparing and determining whether there is an orthogonal relationship between line sets after specific lines are screened.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic devices, and in particular to a route screening method and device, a storage medium and an electronic device. BACKGROUND

[0002] OD (Origin-Destination) refers to the route between the starting grid and the ending grid of a traffic trip. The number of ODs nationwide is at least tens of millions or even hundreds of millions. Each OD has various indicators, for example, the order quantity of the OD in the last 7 days, indicating the order quantity of passengers taking a taxi on the route in the last 7 days; the paired order quantity of the OD in the last 7 days, indicating the order quantity of passengers and drivers on the route in the last 7 days, etc.

[0003] Due to the huge amount of OD data, the ODs are currently stored in HDFS (Hadoop FileSystem, Hadoop file system) in the industry. In order to screen the ODs stored in HDFS, a Hive SQL statement needs to be executed through a Hive computing engine. When an operator wants to obtain some specific ODs, a Hive SQL query statement needs to be written. However, most operators are not developers and many of them do not have computer-related knowledge. Only a small number of senior operators have the ability to write Hive SQL, resulting in a high entry threshold for operators.

[0004] In actual applications, some specific ODs need to be obtained for processing, such as price adjustment. When a certain condition is met, the price is raised; when another condition is met, the price is lowered, etc. However, current methods can only check manually or compare the queried ODs, the former is prone to errors, and the latter is inefficient. SUMMARY

[0005] Embodiments of the present application provide a route screening method, device, storage medium and electronic device, which can improve data query accuracy.

[0006] In a first aspect, embodiments of the present application provide a route screening method, comprising:

[0007] configuring a first rule text, wherein the first rule text is a logical expression composed of one or more conditions;

[0008] performing orthogonal verification of the first rule text and all pre-stored second rule texts, the second rule text being a logical expression composed of one or more conditions;

[0009] storing the first rule text when the verification result indicates that there is no intersection between the first rule text and the second rule text;

[0010] screen a target line meeting the condition from the line database based on the first rule text.

[0011] In a second aspect, the embodiments of the present application provide a line screening device, comprising:

[0012] a configuration unit, configured to configure a first rule text, wherein the first rule text is a logical expression composed of one or more conditions;

[0013] a verification unit, configured to perform orthogonal verification on the first rule text and all pre-stored second rule texts, wherein the second rule text is a logical expression composed of one or more conditions;

[0014] a storage unit, configured to store the first rule text when a verification result indicates that there is no intersection between the first rule text and the second rule text;

[0015] a screening unit, configured to screen a target line meeting the condition from the line database based on the first rule text.

[0016] In an embodiment, the verification unit is configured to:

[0017] perform orthogonal verification according to a first matching rule when the first rule text and the second rule text meet a preset condition;

[0018] perform orthogonal verification according to a second matching rule when the first rule text and the second rule text do not meet the preset condition;

[0019] wherein the preset condition comprises that the outer layer logical operators of the first rule text and the second rule text are both a first preset logical operator, or the outer layer logical operator of the first rule text is the first preset logical operator, the second rule text has no outer layer logical operator, or the first rule text has no outer layer logical operator, and the outer layer logical operator of the second rule text is the first preset logical operator.

[0020] In an embodiment, the screening unit is specifically configured to:

[0021] convert the first rule text into a query statement executable by a specified computing engine;

[0022] screen a target line meeting the condition from the line database according to the query statement.

[0023] In an embodiment, the screening unit is further configured to:

[0024] send the query statement to a containerized cluster management system;

[0025] execute the query statement in parallel based on a plurality of job schedulers under the containerized cluster management system to query the routes matching the query statement from the route database;

[0026] write back the queried routes to the designated route database through the job scheduler;

[0027] read the queried routes from the designated route database as the target routes.

[0028] In an embodiment, the route screening device further comprises:

[0029] a judging unit configured to judge whether there is a second rule text identical to the first rule text;

[0030] a querying unit configured to query the matching routes from the designated route database directly based on the first rule text if the judging unit judges yes.

[0031] In an embodiment, the route screening device further comprises:

[0032] a prompting unit configured to generate prompt information indicating that the first rule text is configured incorrectly when the checking result indicates that the first rule text intersects with the second rule text.

[0033] In an embodiment, the second rule text and the first rule text are stored in the same storage area.

[0034] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, wherein a plurality of instructions are stored in the storage medium, and the instructions are suitable for being loaded by a processor to execute the route screening method described above.

[0035] In a fourth aspect, the embodiments of the present application further provide an electronic device, comprising a processor and a memory, wherein the processor is electrically connected with the memory, the memory is configured to store instructions and data, and the processor is configured to execute the route screening method described above.

[0036] In the embodiments of the present application, the first rule text is configured, and the first rule text is orthogonally checked with all the pre-stored second rule texts; when the checking result indicates that the first rule text does not intersect with the second rule text, the first rule text is stored; and the target routes meeting the conditions are screened from the route database based on the first rule text. The present scheme can improve the data query accuracy. The present scheme determines whether the rule texts are orthogonal before the query, avoids the possibility that the route sets are orthogonal, and improves the route query accuracy without comparing and determining whether the route sets are orthogonal after the specific routes are screened. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to make the technical solutions in the embodiments of the present application clearer, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0038] Figure 1 is a flowchart of a line screening method provided by an embodiment of the present application.

[0039] Figure 2a is a flowchart of a large-scale line OD screening and matching method based on kubernetes provided by an embodiment of the present application.

[0040] Figure 2b is a prediction model architecture diagram provided by an embodiment of the present application.

[0041] Figure 3 is a schematic diagram of a logical expression example provided by an embodiment of the present application.

[0042] Figure 4 is a structural diagram of an electronic device provided by an embodiment of the present application.

[0043] Figure 5 is another structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the scope of protection of the present application.

[0045] When an operator wants to obtain some specific line OD, he needs to write a Hive SQL query statement. However, most operators are not R&D personnel, resulting in a high entry threshold for operators.

[0046] The purpose of obtaining some specific line OD is to process these line ODs, such as price adjustment. That is, when a certain condition is met, the price will be raised; when another condition is met, the price will be lowered, and so on. In practical applications, the processing of line ODs must not both raise and lower the price. Therefore, it is necessary to check whether there is an intersection between the screening conditions to ensure that the OD sets screened under different conditions are mutually exclusive.

[0047] Currently, the Hive SQL query statement execution is one-time, and when exiting the query page, the corresponding Hive SQL query statement is lost, resulting in that the historical query statement cannot be reused.

[0048] In addition, by writing the Hive SQL and performing the line OD query based on the Hive computing engine, each query is at a minute level, resulting in low query efficiency.

[0049] Based on this, the embodiment of the present application provides a line screening method and device, a storage medium and an electronic device, aiming to improve the query accuracy and query efficiency of large-scale line OD. The following will be described in detail.

[0050] In an embodiment, a line screening method is provided, which is applied to electronic devices such as smart phones, tablet computers, notebook computers and the like. Referring to Figure 1 The specific process of the line screening method can be as follows:

[0051] 101, configure a first rule text, wherein the first rule text is a logical expression composed of one or more conditions.

[0052] In the present solution, the language used by the first rule text is human natural language. The logical expression can have only one layer of logic, at most two layers of logic, and each layer is only allowed to have one kind of logical operator, that is, the logical operator of each layer can only be one of "and" and "or", but cannot exist simultaneously. Each condition is composed of three parts: attribute, relationship operator and attribute value. For example, a condition is that the car type is a small van; the car type and age are an attribute; greater than and between are a relationship operator; and the attribute value can be a specific value or a range, such as small van and 18-25 years old. It should be noted that the relationship operator of the text type and the attribute value is slightly different from the numerical type, for example, the car type attribute is a text type, and cannot be said to be greater than a small van.

[0053] In an embodiment, the first text rule can be configured through a web service system, and the user can input a retrieval logical expression through a web front-end webpage, and the web front-end page is limited to ensure the correctness of the input by the operator. For example, the user can input a logical expression through a web page: ((member level is VIP) and (gender is male) and (car type is medium van)).

[0054] 102, orthogonally check the first rule text with all pre-stored second rule texts, the second rule text being a logical expression composed of one or more conditions.

[0055] In the scheme, the orthogonal check refers to checking whether there is an intersection between multiple rule texts. Specifically, the Web business system sends all rule texts to a rule text orthogonal check engine for orthogonal check. The pre-stored second rule text is a historical rule text that has been queried before. After being queried, it can be stored in a limited storage area. For example, after querying a rule text, the rule text can be stored in MySql.

[0056] Due to different user needs, there can be many cases of input rule texts. In order to improve the checking efficiency, the rule texts can be classified, and different checking strategies can be performed for different classes of rule texts. That is, in an embodiment, when the first rule text is orthogonally checked with all pre-stored second rule texts, the following process can be included:

[0057] When the first rule text and the second rule text satisfy the preset condition, the orthogonal check is performed according to the first matching rule;

[0058] When the first rule text and the second rule text do not satisfy the preset condition, the orthogonal check is performed according to the second matching rule;

[0059] The preset condition includes: the outer layer logical operators of the first rule text and the second rule text are both the first preset logical operator, or the outer layer logical operator of the first rule text is the first preset logical operator, the second rule text does not have an outer layer logical operator, or the first rule text does not have an outer layer logical operator, and the outer layer logical operator of the second rule text is the first preset logical operator.

[0060] For convenience of description, a certain rule is described as A rule, another rule is described as B rule, a certain one-layer logical expression of the A rule is described as a one-layer logical expression, another one-layer logical expression of the B rule is described as β one-layer logical expression, and so on.

[0061] Specifically, when the outer layer logical operator of the A rule is "and" and the outer layer logical operator of the B rule is "and", or the outer layer logical operator of the A rule is "and" and the B rule does not have an outer layer logical operator, or the A rule does not have an outer layer logical operator and the outer layer logical operator of the B rule is "and", the rule text full matching is performed, otherwise the rule text arbitrary matching is performed. The specific steps are as follows:

[0062] Rule text full matching stage a1:

[0063] If the A rule and the B rule have at least one same attribute, the next stage a2 is entered. Otherwise, the A rule and the B rule do not have orthogonality.

[0064] Rule text full match phase a2:

[0065] Find the one-layer logical expressions with the same attribute in A rule and B rule. Compare the one-layer logical expressions with the same attribute two by two, and determine whether the one-layer logical expression with the same attribute is to enter the condition full match phase c or the condition arbitrary match phase d, until all one-layer logical expressions with the same attribute are compared. If the inner logical operator of the one-layer logical expression of a in A rule is "and" and the inner logical operator of the one-layer logical expression of β in B rule is "and", or if the inner logical operator of the one-layer logical expression of a in A rule is "and" and the one-layer logical expression of β in B rule does not exist inner logical operator, or if the one-layer logical expression of a in A rule does not exist inner logical operator and the inner logical operator of the one-layer logical expression of β in B rule is "and", then enter the condition full match phase c. Otherwise, enter the condition arbitrary match phase d.

[0066] Rule text full match phase a3:

[0067] Wait for the results returned by the condition full match phase c or the condition arbitrary match phase d. A rule and B rule exist orthogonally if and only if all results are true, otherwise A rule and B rule do not exist orthogonally.

[0068] Rule text arbitrary match phase b1:

[0069] If A rule and B rule exist at least one same attribute, then enter the next phase b2. Otherwise, A rule and B rule must exist orthogonally.

[0070] Rule text arbitrary match phase b2:

[0071] Find the one-layer logical expressions with the same attribute in A rule and B rule. Compare the one-layer logical expressions with the same attribute two by two, and determine whether the one-layer logical expression with the same attribute is to enter the condition full match phase c or the condition arbitrary match phase d, until all one-layer logical expressions with the same attribute are compared. If the inner logical operator of the one-layer logical expression of a in A rule is "and" and the inner logical operator of the one-layer logical expression of β in B rule is "and", or if the inner logical operator of the one-layer logical expression of a in A rule is "and" and the one-layer logical expression of β in B rule does not exist inner logical operator, or if the one-layer logical expression of a in A rule does not exist inner logical operator and the inner logical operator of the one-layer logical expression of β in B rule is "and", then enter the condition full match phase c. Otherwise, enter the condition arbitrary match phase d.

[0072] Rule text arbitrary match phase b3:

[0073] The result returned by the condition full match stage c or the condition any match stage d. When any of the returned results is true, the A rule and the B rule exist orthogonally; otherwise, the A rule and the B rule do not exist orthogonally.

[0074] The condition full match stage c:

[0075] Find the conditions with the same attribute of the α-level logical expression and the β-level logical expression, compare the conditions with the same attribute two by two, and determine whether the attribute values exist orthogonally until all the conditions with the same attribute are compared. When and only when the conditions with the same attribute do not exist orthogonally, the α-level logical expression and the β-level logical expression are not orthogonal, at this time, record the result of this condition full match stage c as false, and return the result to the entered rule text matching stage, that is, if this condition full match stage c is entered by the rule text full match stage a2, return to the rule text full match stage a3, if this condition full match stage c is entered by the rule text any match stage b2, return to the rule text any match stage b3; otherwise, the α-level logical expression and the β-level logical expression exist orthogonally, at this time, record the result of this condition full match stage c as true, and return the result to the entered rule text matching stage, that is, if this condition full match stage c is entered by the rule text full match stage a2, return to the rule text full match stage a3, if this condition full match stage c is entered by the rule text any match stage b2, return to the rule text any match stage b3.

[0076] The condition any match stage d:

[0077] The condition that the alpha one layer logic expression and the beta one layer logic expression have the same attribute is found out, the conditions having the same attribute are compared with each other, and it is determined whether there is orthogonality between the attribute values, until all the conditions having the same attribute are compared. When there is orthogonality between any condition having the same attribute, the alpha one layer logic expression and the beta one layer logic expression have orthogonality, at this time, the result of the present condition arbitrary matching stage d is recorded as true, and the result is returned to the entered rule text matching stage, that is, if the present condition arbitrary matching stage d is entered by the rule text full matching stage a2, the rule text full matching stage a3 is returned to, if the present condition arbitrary matching stage d is entered by the rule text arbitrary matching stage b2, the rule text arbitrary matching stage b3 is returned to; otherwise, the alpha one layer logic expression and the beta one layer logic expression do not have orthogonality, at this time, the result of the present condition arbitrary matching stage d is recorded as false, and the result is returned to the entered rule text matching stage, that is, if the present condition arbitrary matching stage d is entered by the rule text full matching stage a2, the rule text full matching stage a3 is returned to, if the present condition arbitrary matching stage d is entered by the rule text arbitrary matching stage b2, the rule text arbitrary matching stage b3 is returned to.

[0078] In order to avoid repeated queries and save device resources, after the first rule text is configured, it can be determined whether there is a second rule text which is the same as the first rule text. If there is, the matching line is directly queried from the specified line database based on the first rule text. In the present scheme, the line OD in the specified line database is the line OD written from a more bottom layer storage area to the current storage area in advance. It should be noted that the specified line database has fast reading speed, can allow the business end to modify its data, and can respond to the demand of the business end in time. In the present scheme, the specified line database can be Hbase.

[0079] 103、When the checking result indicates that there is no intersection between the first rule text and the second rule text, the first rule text is stored.

[0080] In the present scheme, when it is determined that there is no intersection between the first rule text and the second rule text, the first rule text is stored, and the second rule text and the first rule text can be stored in the same storage area. When the checking result indicates that there is intersection between the first rule text and the second rule text, prompt information indicating that the first rule text is configured incorrectly is generated.

[0081] For example, when the rule text orthogonality checking engine determines that the first rule text and the second rule text are not orthogonal, the rule text is written into MySql for storage, and the rule text is sent to the rule text to Hive SQL engine; otherwise, it is ended, and the Web business system is prompted that there is orthogonality between the rule texts.

[0082] 104. filtering target lines meeting the condition from the line database based on the first rule text.

[0083] In the scheme, the line database is HDFS. The underlying of Hive and Hbase is HDFS, and both need the support of HDFS. Without HDFS, Hive and Hbase cannot run. Hive is essentially a computing engine, contains a meta table, and provides a mapping relationship, but does not store data itself; Hbase is a database that stores data itself, but the data stored by itself is stored in the underlying HDFS.

[0084] In an embodiment, filtering target lines meeting the condition from the line database based on the first rule text comprises the following process:

[0085] converting the first rule text into a query statement executable by a specified computing engine;

[0086] filtering target lines meeting the condition from the line database according to the query statement.

[0087] In the scheme, the rule text can be converted into a Hive SQL engine, and the rule text in the format of human natural language is converted into a SQL statement that can be used for Hive query. In the rule text to Hive SQL engine, the mapping relationship between the attributes and the Hive table fields, the mapping relationship between the relationship operators and the relationship operators of the Hive SQL query statement, and the mapping relationship between the logical operators and the logical operators of the Hive SQL query statement are stored in advance.

[0088] Specifically, the outer logical operator is obtained and converted into the logical operator of the Hive SQL query statement, and recorded as logic-1. Then, one layer of logical expression is looped, each one layer of logical expression is parsed, and a plurality of conditions and inner logical operators logic-2 are obtained. Then, each condition is looped, the attributes and the relationship operators in the condition are converted into the Hive table fields and the Hive SQL query statement relationship operators, and the plurality of converted conditions are connected using logic-2 to obtain the Hive SQL query statement corresponding to one layer of logical expression. Finally, using logic-1, all one layer of logical expression corresponding to the Hive SQL query statement obtained after the loop is connected. The obtained Hive SQL query statement is added with the table name of the select query to obtain the complete Hive SQL query statement, realizing the conversion from human natural language to Hive SQL query statement.

[0089] In an embodiment, filtering target lines meeting the condition from the line database according to the query statement can comprise the following process:

[0090] sending the query statement to a containerized cluster management system;

[0091] executing the query statement in parallel based on multiple job schedulers under the containerized cluster management system to query routes matching the query statement from the route database;

[0092] writing the queried routes back to the designated route database through the job scheduler;

[0093] reading the queried routes from the designated route database as target routes.

[0094] Specifically, the containerized cluster management system can be kubernetes, and the job scheduler can be Job under kubernetes. Multiple Job tasks of kubernetes are started, and SQL statements available for Hive query are sent to the multiple Job, and the query of Hive SQL is executed to realize distributed concurrency and improve query performance. The Job can write the queried route OD back to Hbase, and the web business system can query the route OD stored in Hbase to finally filter out the desired target route.

[0095] As can be seen from the above, the route filtering method provided in the embodiment filters out the target route meeting the condition from the route database based on the first rule text, and stores the first rule text when the verification result indicates that there is no intersection between the first rule text and the second rule text. The scheme can improve the data query accuracy. The scheme determines whether the rule texts are orthogonal before query, avoids the possibility that the route sets are orthogonal, and improves the route query accuracy without comparing and determining whether the route sets are orthogonal after filtering out the specific route.

[0096] In another embodiment of the present application, a large-scale route OD filtering and matching method based on kubernetes is also provided. Referring to Figure 2a , the route OD filtering and matching method provided will be described in detail below. Specifically as follows:

[0097] (I) The web business system configures a rule text and compares it with the rule text stored in MySql.

[0098] Specifically, if the same rule text already exists in MySql, the route OD is directly queried from Hbase, otherwise the next step is entered.

[0099] (II) Determine whether the rule texts are orthogonal through a rule text orthogonal engine.

[0100] Specifically, the web business system sends all rule text to the rule text orthogonal validation engine. Rule text is a logical expression consisting of one or more conditions. A logical expression can have only one level of logic, but at most two levels, and each level is allowed only one logical operator; that is, each level's logical operator can only be either "AND" or "OR," and cannot exist simultaneously. For example, `((Member level is VIP) AND (Gender is male) AND (Vehicle type is a medium-sized van))` conforms to the rule; while `((Member level is VIP) AND (Gender is male) OR (Vehicle type is a medium-sized van))` does not conform to the rule. When a logical expression has only one condition, it can omit logical operators, such as `((Member level is VIP))`. This restriction is implemented through the web front-end page to ensure the correctness of input by operations personnel. (Reference) Figure 2b , Figure 2b The configured rule text is indistinguishable from natural language used by humans.

[0101] In this embodiment, the conditions constituting the logical expression consist of three parts: attributes, relational operators, and attribute values. In this scheme, orthogonality refers to whether multiple rule texts intersect, which can be determined by a rule text orthogonality validation engine. For example: `((Member level is VIP) and (Age is greater than 18 years old))` and `((Member level is VIP) and (Age is greater than 30 years old))`. Since "greater than 18 years old" encompasses "greater than 30 years old" in the age attribute, there is orthogonality between the two.

[0102] For ease of description, let's describe one rule as rule A and another as rule B. Let's describe one level of logical expression in rule A as an α-level logical expression, another level of logical expression in rule B as a β-level logical expression, and so on. The algorithm used by the rule text orthogonality engine to determine whether rules are orthogonal is as follows:

[0103] If the outer logical operator of rule A is "AND" and the outer logical operator of rule B is "AND", or if the outer logical operator of rule A is "AND" and rule B does not have an outer logical operator, or if rule A does not have an outer logical operator and rule B has an outer logical operator "AND", then perform a full match of the rule text; otherwise, perform an arbitrary match of the rule text.

[0104] (iii) If the rules are not orthogonal, store the rule text in MySQL and send it to the Hive SQL engine. Otherwise, end the process and indicate that the rules are orthogonal.

[0105] Specifically, if the rule text is determined to be non-orthogonal by the rule text orthogonality checking engine, the rule text is written into the MySql for storage, and the rule text is sent to the rule text to Hive SQL engine; otherwise, the process is ended, and the Web business system is prompted that the rule text is orthogonal.

[0106] (IV) The rule text to Hive SQL engine converts the rule text in human natural language into a query statement that can be used for Hive SQL execution.

[0107] In the present scheme, the rule text to Hive SQL engine is used to convert the rule text in human natural language format into a SQL statement that can be used for Hive query.

[0108] In the rule text to Hive SQL engine, the mapping relationship between the attributes and the Hive table fields, the mapping relationship between the relationship operators and the relationship operators of the Hive SQL query statement, and the mapping relationship between the logical operators and the logical operators of the Hive SQL query statement are stored. In specific implementation, the following steps are included:

[0109] (11) The outer logical operator is obtained, converted into the logical operator of the Hive SQL query statement, and temporarily cached, and recorded as logic-1;

[0110] (21) A one-layer logical expression is looped, each one-layer logical expression is parsed, and a plurality of conditions and an inner logical operator logic-2 are obtained;

[0111] (22) Each condition is looped, the attributes and the relationship operators in the condition are converted into the Hive table fields and the relationship operators of the Hive SQL query statement, and the plurality of converted conditions are connected using the logic-2 to obtain the Hive SQL query statement corresponding to the one-layer logical expression;

[0112] (23) All one-layer logical expression corresponding Hive SQL query statements obtained after the loop are connected using the logic-1;

[0113] (24) The obtained Hive SQL query statement is added with the table name of the select of the query. The complete Hive SQL query statement is obtained.

[0114] For example, the rule text: ((vehicle type is a small van) and (age between 18-25 years old)) or ((membership level is VIP) and (gender is male) and (vehicle type is a medium van or a large truck)) or ((gender is female) or (age is over 30 years old)) is a logical expression containing two levels of logic. Using the Hive SQL engine, this rule text can be converted into the following SQL query statement:

[0115] Select * from table_name where

[0116] ((vehicle = 'small bread')and(age >= 25))

[0117] or

[0118] (level='VIP')and(gender='Male')and(vehicle in('Minivan','Large Truck'))

[0119] or

[0120] ((gender='female')or(age>30))

[0121] (v) By sending Hive SQL query statements to multiple jobs in Kubernetes for execution.

[0122] Specifically, multiple Kubernetes jobs are started, and SQL statements usable for Hive queries are sent to these jobs. The Hive SQL queries are then executed by these jobs, achieving distributed concurrency. Each job executes only one Hive SQL query statement converted from the rule text. If there are n Hive SQL queries converted from the rule text, then n jobs are started and executed simultaneously. Assuming that executing one Hive SQL query statement takes time *t*, the total execution time in the previous approach was *t*n; when using multiple jobs for distributed execution, the time consumed is only *t*.

[0123] (vi) Write back the OD retrieved by the Job to HBase.

[0124] In practical applications, if the route OD under rule A is not stored in HBase, the web application system will not be able to query it. Therefore, it is necessary to first use a Job to pull the route OD from Hive and write it to HBase according to rule A, so that the web application system can query the OD according to rule A.

[0125] (vii) Notify the Web business system to query OD from HBase.

[0126] Specifically, by writing the queried OD data back to Hbase, due to the real-time performance of Hbase, the OD in Hbase is queried through the Web business system, and finally the expected OD is obtained, so that the query efficiency is greatly improved.

[0127] The product sales prediction method provided in the embodiments of the application can use human natural language to query the OD stored in HDFS, thereby reducing the entry threshold of users.

[0128] Using the rule text orthogonality engine to determine whether the logical expression exists orthogonality can avoid the possibility that the OD sets exist orthogonality before querying, without waiting for specific ODs to be screened out and then manually comparing and determining whether the OD sets exist orthogonality, thereby saving the labor cost and reducing the human efficiency.

[0129] Based on the parallel execution of multiple Jobs of kubernetes, the time-consuming query of Hive SQL is distributed to multiple Jobs for parallel execution, thereby greatly shortening the query time and improving the query efficiency.

[0130] Hbase is introduced, and the queried OD is written back to Hbase through a Job. Due to the real-time performance of Hbase, under the same filtering condition, the first query is implemented at a second level (even if the Hive SQL query is executed through multiple Jobs, it can only be reduced to a second level, and cannot reach a real-time level), and other subsequent queries are implemented at a real-time level. Finally, the problem of slow query efficiency is solved.

[0131] In another embodiment of the application, a line screening device is also provided. The line screening device can be integrated in an electronic device in the form of software or hardware. The electronic device can specifically include a mobile phone, a tablet computer, a notebook computer, and the like. As shown in Figure 3 The line screening device 300 can include a configuration unit 301, a verification unit 302, a storage unit 303, and a screening unit 304.

[0132] The configuration unit 301 is configured to configure a first rule text, wherein the first rule text is a logical expression composed of one or more conditions.

[0133] The verification unit 302 is configured to perform orthogonality verification on the first rule text and all pre-stored second rule texts, wherein the second rule text is a logical expression composed of one or more conditions.

[0134] The storage unit 303 is configured to store the first rule text when the verification result indicates that the first rule text and the second rule text do not have an intersection.

[0135] The screening unit 304 is configured to screen a target line meeting a condition from the line database based on the first rule text.

[0136] In an embodiment, the checking unit 302 is configured to:

[0137] When the first rule text and the second rule text meet a preset condition, performing the orthogonal checking according to a first matching rule;

[0138] When the first rule text and the second rule text do not meet the preset condition, performing the orthogonal checking according to a second matching rule;

[0139] The preset condition includes that the outer layer logical operators of the first rule text and the second rule text are both a first preset logical operator, or the outer layer logical operator of the first rule text is the first preset logical operator, the second rule text does not have an outer layer logical operator, or the first rule text does not have an outer layer logical operator, and the outer layer logical operator of the second rule text is the first preset logical operator.

[0140] In an embodiment, the screening unit 304 is specifically configured to:

[0141] convert the first rule text into a query statement executable by a specified computing engine;

[0142] screen the target line meeting the condition from the line database according to the query statement.

[0143] In an embodiment, the screening unit 304 is further configured to:

[0144] send the query statement to a containerized cluster management system;

[0145] parallelly execute the query statement based on a plurality of job schedulers under the containerized cluster management system to query a line matching the query statement from the line database;

[0146] write the queried line back to a specified line database through the job scheduler;

[0147] read the queried line from the specified line database as the target line.

[0148] In an embodiment, the line screening apparatus 300 can further include:

[0149] a judging unit configured to judge whether there is a second rule text identical to the first rule text;

[0150] a querying unit configured to, if the judging unit judges yes, directly query a matching line from the specified line database based on the first rule text.

[0151] In an embodiment, the line screening device 300 can further include:

[0152] The prompting unit is configured to generate prompt information indicating that the first rule text is configured incorrectly when the checking result indicates that the first rule text and the second rule text have an intersection.

[0153] In an embodiment, the second rule text and the first rule text are stored in the same storage area.

[0154] As can be seen from the above, the line screening device provided by the embodiments of the present application can configure a first rule text, and perform orthogonal checking on the first rule text and all pre-stored second rule texts; when a checking result indicates that the first rule text and the second rule text have no intersection, the first rule text is stored; and based on the first rule text, a target line meeting a condition is screened out from a line database. The present scheme can improve data query accuracy. The present scheme determines whether the rule texts have an orthogonal relationship before query, avoids the possibility that there is an orthogonal relationship between line sets, and does not need to compare and determine whether there is an orthogonal relationship between line sets after specific lines are screened out, thereby improving line query accuracy.

[0155] In another embodiment of the present application, an electronic device is also provided. The electronic device can be a smart phone, a tablet computer, or the like. As shown in Figure 4 The electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0156] The processor 401 is the control center of the electronic device 400, and connects all parts of the electronic device through various interfaces and lines. The processor 401 executes various functions of the electronic device and processes data by running or loading applications stored in the memory 402 and calling data stored in the memory 402, thereby overall monitoring the electronic device.

[0157] In the present embodiment, the processor 401 in the electronic device 400 loads the instructions corresponding to the processes of one or more applications into the memory 402, and runs the applications stored in the memory 402 by the processor 401, thereby realizing various functions in the following steps:

[0158] Configuring a first rule text, wherein the first rule text is a logical expression composed of one or more conditions;

[0159] Orthogonal checking the first rule text and all pre-stored second rule texts, wherein the second rule text is a logical expression composed of one or more conditions;

[0160] store the first rule text when the check result indicates that the first rule text and the second rule text have no intersection;

[0161] filter target lines meeting the condition from the line database based on the first rule text.

[0162] In an embodiment, when performing the orthogonal check on the first rule text and all the pre-stored second rule texts, the processor 401 can perform the following operations:

[0163] perform the orthogonal check according to a first matching rule when the first rule text and the second rule text meet a preset condition;

[0164] perform the orthogonal check according to a second matching rule when the first rule text and the second rule text do not meet the preset condition;

[0165] The preset condition includes that the outer layer logical operators of the first rule text and the second rule text are both a first preset logical operator, or the outer layer logical operator of the first rule text is the first preset logical operator, the second rule text has no outer layer logical operator, or the first rule text has no outer layer logical operator, and the outer layer logical operator of the second rule text is the first preset logical operator.

[0166] In an embodiment, when filtering target lines meeting the condition from the line database based on the first rule text, the processor 401 can perform the following operations:

[0167] convert the first rule text into a query statement executable by a specified computing engine;

[0168] filter target lines meeting the condition from the line database according to the query statement.

[0169] In an embodiment, when filtering target lines meeting the condition from the line database according to the query statement, the processor 401 can perform the following operations:

[0170] send the query statement to a containerized cluster management system;

[0171] parallelly execute the query statement based on multiple job schedulers under the containerized cluster management system to query lines matching the query statement from the line database;

[0172] write the queried lines back to a specified line database through the job schedulers;

[0173] read the queried lines from the specified line database as the target lines.

[0174] In an embodiment, after the first rule text is configured, the processor 401 can further perform the following operations before performing the orthogonal check of the first rule text with the pre-stored second rule text:

[0175] determining whether there is a second rule text identical to the first rule text;

[0176] if yes, directly querying the matching route from the specified route database based on the first rule text.

[0177] In an embodiment, the processor 401 can be further configured to perform the following operations:

[0178] generating prompt information indicating that the first rule text is configured incorrectly when the check result indicates that the first rule text intersects with the second rule text.

[0179] The memory 402 can be configured to store applications and data. The applications stored in the memory 402 include instructions executable in the processor. The applications can constitute various functional modules. The processor 401 executes various functional applications and route screening by running the applications stored in the memory 402.

[0180] In some embodiments, as shown in FIG. 4, the electronic device 400 further includes a display screen 403, a control circuit 404, a radio frequency circuit 405, an input unit 406, a sensor 408, and a power supply 409. The processor 401 is electrically connected to the display screen 403, the control circuit 404, the radio frequency circuit 405, the input unit 406, the camera 407, the sensor 408, and the power supply 409, respectively. Figure 5 The display screen 403 can be configured to display information input by a user or information provided to a user, as well as various graphical user interfaces of the electronic device, which can be composed of images, text, icons, videos, and any combination thereof.

[0181] The control circuit 404 is electrically connected to the display screen 403 and is configured to control the display screen 403 to display information.

[0182] The radio frequency circuit 405 is configured to transceive radio frequency signals to establish wireless communication with an electronic device or other electronic devices, and transceive signals between a server or other electronic devices.

[0183] The input unit 406 can be configured to receive inputted digital, character information, or user feature information (e.g., fingerprint), and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The input unit 406 can include a fingerprint recognition module.

[0184]

[0185] ​The camera 407 can be used to collect image information. The camera can be a single camera with one lens, or can have two or more lenses.

[0186] The sensor 408 is used to collect external environment information. The sensor 408 can include an ambient brightness sensor, an acceleration sensor, a light sensor, a motion sensor, and other sensors.

[0187] The power supply 409 is used to supply power to various components of the electronic device 400. In some embodiments, the power supply 409 can be logically connected to the processor 401 through a power management system, so that the power management system can manage charging, discharging, and power consumption management, etc.

[0188] Although Figure 5 The electronic device 400 can also include a speaker, a Bluetooth module, etc., which are not shown in the figure.

[0189] As can be seen from the above, the electronic device provided by the embodiments of the present application can configure a first rule text, and perform orthogonal checking on the first rule text and all pre-stored second rule texts. When the checking result indicates that there is no intersection between the first rule text and the second rule text, the first rule text is stored. The target line that meets the condition is filtered out from the line database based on the first rule text. The present scheme can improve the data query accuracy. The present scheme determines whether the rule text is orthogonal before querying, avoids the possibility that there is an orthogonal between the line sets, and does not need to compare and determine whether there is an orthogonal between the line sets after filtering out the specific line, thereby improving the line query accuracy.

[0190] In some embodiments, a computer readable storage medium is also provided, and the storage medium stores a plurality of instructions adapted to be loaded by a processor to execute any of the above line filtering methods.

[0191] Those skilled in the art can understand that all or part of the steps in the above embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, which can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0192] The line screening method and device provided by the embodiments of the present application, the storage medium and the electronic device are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. A method for selecting circuits, characterized in that, include: Configure the first rule text, wherein the first rule text is a logical expression consisting of one or more conditions, and the language used in the first rule text is human natural language; Before querying the route, the first rule text is orthogonally validated against all pre-stored second rule texts. The second rule text is a logical expression consisting of one or more conditions. When the verification result indicates that the first rule text and the second rule text have no intersection, store the first rule text; Based on the first rule text, target routes that meet the conditions are filtered from the route database. The step of filtering target routes that meet the conditions from the route database based on the first rule text includes: The first rule text is converted into a query statement that can be executed by the specified computing engine. Specifically, the first rule text, which uses human natural language, is converted into a Hive SQL query statement that can be used for Hive queries. In the rule text to Hive SQL engine conversion, the mapping relationship between attributes and Hive table fields, the mapping relationship between relational operators and Hive SQL query statement relational operators, and the mapping relationship between logical operators and Hive SQL query statement logical operators are pre-stored. Based on the query statement, select target routes that meet the conditions from the route database; The step of filtering target routes that meet the conditions from the route database according to the query statement includes: Send the query statement to the containerized cluster management system; The query statement is executed in parallel by multiple job schedulers under the containerized cluster management system to query the line database for lines that match the query statement. The job scheduler writes the queried lines back to the designated line database. The queried route is read from the specified route database and used as the target route.

2. The method according to claim 1, characterized in that, The step of performing orthogonal verification between the first rule text and all pre-stored second rule texts includes: When the first rule text and the second rule text meet the preset conditions, orthogonal verification is performed according to the first matching rule; When the first rule text and the second rule text do not meet the preset conditions, orthogonal verification is performed according to the second matching rule. The preset conditions include: the outer logical operators of the first rule text and the second rule text are both the first preset logical operators, or the outer logical operator of the first rule text is the first preset logical operator and the second rule text does not have an outer logical operator, or the outer logical operator of the first rule text does not have an outer logical operator and the outer logical operator of the second rule text is the first preset logical operator.

3. The method according to claim 1, characterized in that, After configuring the first rule text and before performing orthogonal verification between the first rule text and the pre-stored second rule text, the following steps are also included: Determine if there exists a second rule text that is identical to the first rule text; If so, then the matching line is directly queried from the specified line database based on the first rule text.

4. The method according to claim 1, characterized in that, Also includes: When the verification result indicates that the first rule text and the second rule text have an intersection, a prompt message is generated to indicate that the first rule text is configured incorrectly.

5. The method according to any one of claims 1-4, characterized in that, The second rule text is stored in the same storage area as the first rule text.

6. A circuit screening device, characterized in that, include: The configuration unit is used to configure the first rule text, wherein the first rule text is a logical expression composed of one or more conditions, and the language used in the first rule text is human natural language; The verification unit is used to perform orthogonal verification between the first rule text and all pre-stored second rule texts before querying the line. The second rule text is a logical expression consisting of one or more conditions. A storage unit is used to store the first rule text when the verification result indicates that the first rule text and the second rule text have no intersection; The filtering unit is used to filter target routes that meet the conditions from the route database based on the first rule text. The step of filtering target routes that meet the conditions from the route database based on the first rule text includes: The first rule text is converted into a query statement that can be executed by the specified computing engine. Specifically, the first rule text, which uses human natural language, is converted into a Hive SQL query statement that can be used for Hive queries. In the rule text to Hive SQL engine conversion, the mapping relationship between attributes and Hive table fields, the mapping relationship between relational operators and Hive SQL query statement relational operators, and the mapping relationship between logical operators and Hive SQL query statement logical operators are pre-stored. Based on the query statement, select target routes that meet the conditions from the route database; The step of filtering target routes that meet the conditions from the route database according to the query statement includes: Send the query statement to the containerized cluster management system; The query statement is executed in parallel by multiple job schedulers under the containerized cluster management system to query the line database for lines that match the query statement. The job scheduler writes the queried lines back to the designated line database. The queried route is read from the specified route database and used as the target route.

7. A computer-readable storage medium, characterized in that, The storage medium stores a plurality of instructions, which are adapted to be loaded by a processor to execute the line screening method according to any one of claims 1-5.

8. An electronic device, characterized in that, The device includes a processor and a memory, wherein the processor is electrically connected to the memory, and the memory is used to store instructions and data; the processor is used to execute the circuit screening method according to any one of claims 1-5.

Images