Data screening method, device and equipment and storage medium

Abstract

The application discloses a data screening method and device, equipment and a storage medium, which can be widely applied to the field of artificial intelligence technology. The method comprises the following steps: obtaining original data, performing data processing on the original data, and obtaining label data; creating target rule data corresponding to the label data according to the label data, wherein at least two target screening rules are stored in the target rule data; obtaining serial number data corresponding to each target screening rule, performing data combination processing on the serial number data, and obtaining rule combination data; obtaining a target screening condition, performing data screening processing on the original data according to the rule combination data and the target screening condition, and obtaining target screening data. The embodiment of the application can effectively improve the data screening efficiency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence, and in particular to a data screening method and device, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI), the data screening work is gradually freed from manual work and realized intelligent. For large amount of data screening, in the related technology, it is usually screened through a database or a demand is proposed to assist IT personnel, but the above-mentioned methods need to consume a lot of time, resulting in low data screening efficiency. SUMMARY

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The main purpose of the embodiments of the present application is to provide a data screening method, device, equipment and storage medium, which can effectively improve the data screening efficiency.

[0005] To achieve the above-mentioned purpose, in a first aspect, the embodiments of the present application provide a data screening method, comprising:

[0006] Obtaining original data, performing data processing on the original data to obtain label data;

[0007] According to the label data, creating target rule data corresponding to the label data, wherein the target rule data stores at least two target screening rules;

[0008] Obtaining serial number data corresponding to each target screening rule, performing data combination processing on the serial number data to obtain rule combination data;

[0009] Obtaining target screening conditions, performing data screening processing on the original data according to the rule combination data and the target screening conditions to obtain target screening data.

[0010] In some embodiments, the target rule data is provided with at least two of a one-to-one corresponding label item, an operator item, a screening condition item, a time range item, a comparison value item and a numerical type item, and after the target rule data corresponding to the label data is created, the method comprises:

[0011] Obtaining target screening rules, performing data splitting processing on the target screening rules to obtain splitting data corresponding to the target screening rules, wherein the splitting data corresponds to the serial number data;

[0012] In a case where the split data includes a target label, the target label is stored in the label item of the target rule data;

[0013] In a case where the split data includes a target operator, the target operator is stored in the operator item of the target rule data;

[0014] In a case where the split data includes a target filter condition, the target filter condition is stored in the filter condition item of the target rule data;

[0015] In a case where the split data includes a target time range, the target time range is stored in the time range item of the target rule data;

[0016] In a case where the split data includes a target comparison value, the target comparison value is stored in the comparison value item of the target rule data;

[0017] In a case where the split data includes a target numerical value type, the target numerical value type is stored in the numerical value type item of the target rule data.

[0018] In some embodiments, the data combination processing on the serial number data includes:

[0019] The data combination processing on the serial number data is performed according to a preset execution instruction, and the preset execution instruction includes a preset execution sequence instruction and / or a preset execution condition instruction.

[0020] In some embodiments, after the target filter condition is obtained, the method includes:

[0021] The target filter condition is subjected to rule split processing according to the label data, to obtain a target to-be-filtered rule corresponding to the target filter condition;

[0022] The target to-be-filtered rule is subjected to the data combination processing, to obtain rule target combination data.

[0023] In some embodiments, the data filtering processing on the original data according to the rule combination data and the target filter condition includes:

[0024] In a case where the rule combination data includes the rule target combination data, the data filtering processing on the original data is performed according to the rule target combination data, to obtain target filtering data.

[0025] In some embodiments, the data screening processing of the original data according to the rule combination data and the target screening condition comprises:

[0026] In the case that the rule combination data does not include the rule target combination data, the rule test instruction corresponding to the rule target combination data is acquired;

[0027] According to the rule test instruction, the rule target combination data is matched with the target rule data and the rule combination data to obtain matching data;

[0028] When the matching data satisfies a preset matching condition, the original data is screened according to the rule target combination data to obtain target screening data.

[0029] In some embodiments, the data screening processing of the original data according to the rule target combination data comprises:

[0030] The original data is screened according to the rule target combination data to obtain initial screening data;

[0031] The initial screening data is subjected to data summarization processing and data deduplication processing to obtain the target screening data.

[0032] In a second aspect, the embodiments of the present application provide a data screening device, comprising:

[0033] A data processing module is configured to acquire original data, and perform data processing on the original data to obtain label data;

[0034] A data creating module is configured to create target rule data corresponding to the label data according to the label data, wherein the target rule data stores at least two target screening rules;

[0035] A data combination module is configured to acquire serial number data corresponding to each target screening rule, and perform data combination processing on the serial number data to obtain rule combination data;

[0036] A data screening module is configured to acquire a target screening condition, and perform data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data.

[0037] In a third aspect, the embodiments of the present application provide a data screening device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data screening method of the foregoing embodiments when executing the computer program.

[0038] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer executable program for executing the data screening method of the above embodiment.

[0039] The beneficial effects of the embodiments of the present application include: obtaining original data, performing data processing on the original data to obtain label data; creating target rule data corresponding to the label data according to the label data, wherein the target rule data stores at least two target screening rules; obtaining serial number data corresponding to each target screening rule, performing data combination processing on the serial number data to obtain rule combination data; obtaining a target screening condition, performing data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data. Compared with related technologies, the embodiments of the present application can realize fast screening of the original data according to the rule combination data and the target screening condition for the target screening condition to be screened in actual application, and then obtain more accurate target screening data. The embodiments of the present application can effectively improve the data screening efficiency.

[0040] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings are intended to provide a further understanding of the technical solutions of the present application and constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0042] Figure 1 A flowchart of the data screening method of the embodiment of the present application is shown in FIG. 1;

[0043] Figure 2 A flowchart of the split data of the embodiment of the present application is shown in FIG. 2;

[0044] Figure 3 A flowchart of the rule combination data of the embodiment of the present application is shown in FIG. 3;

[0045] Figure 4 A flowchart of the rule target combination data of the embodiment of the present application is shown in FIG. 4;

[0046] Figure 5 A flowchart of the target screening data of the embodiment of the present application is shown in FIG. 5;

[0047] Figure 6 A flowchart of the target screening data of another embodiment of the present application is shown in FIG. 6;

[0048] Figure 7 Flowchart of the process of screening data for another embodiment of the present application;

[0049] Figure 8 Structure diagram of the data screening device for the embodiment of the present application;

[0050] Figure 9 Hardware structure diagram of the data screening device for the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0052] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0054] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a sufficient understanding of the embodiments of the present disclosure. However, one skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring the aspects of the present disclosure.

[0055] The block diagram shown in the drawings is only a functional entity, which does not necessarily correspond to a physically independent entity. That is, these functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0056] The flowcharts shown in the drawings are only illustrative, and are not necessarily inclusive of all content and operations / steps, nor are they necessarily performed in the order described. For example, some operations / steps can be broken down, and some operations / steps can be combined or partially combined, so the actual order of performance can be changed according to actual conditions.

[0057] First, some terms involved in the present application are analyzed:

[0058] Artificial Intelligence (AI): is a new technical science of researching and developing theory, method, technology and application system for simulating, extending and expanding human intelligence; Artificial Intelligence is a branch of computer science, and Artificial Intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems, etc. Artificial Intelligence can simulate the information process of human consciousness and thinking. Artificial Intelligence is also the theory, method, technology and application system of using digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] ERS (Enterprises Run System): is the ultimate form of enterprise management control software system after the ERP (Enterprise Resource Planning) system, and has good market prospects.

[0060] ID (Identity Document): is the abbreviation of identity card identification number, account number, unique code, exclusive number, industrial design, legal vocabulary, general account, decoder, software company and various proprietary vocabulary.

[0061] With the rapid development of Artificial Intelligence (AI), the screening of data has gradually been freed from manual work and realized intelligentization. For large quantities of data screening, in related technologies, it is usually screened through a database or the demand is proposed to IT personnel for assistance, but the above-mentioned ways all need to consume a lot of time, resulting in low data screening efficiency.

[0062] For example, when a user analyst analyzes the industrial performance, renewal rate, and cancellation rate, etc., a large amount of data screening is often required. At this time, whether it is screened through a database by oneself or the demand is proposed to IT personnel for assistance to write scripts or set up a timing task to screen, a lot of time is consumed, and the time of test personnel and business personnel is also occupied when verifying the correctness of the data, resulting in low data screening efficiency.

[0063] Based on this, the embodiment of the application provides a data screening method, device and equipment and a storage medium. The original data is obtained, and data processing is performed on the original data to obtain label data. Target rule data corresponding to the label data is created according to the label data, wherein at least two target screening rules are stored in the target rule data. Serial number data corresponding to each target screening rule is obtained, and data combination processing is performed on the serial number data to obtain rule combination data. A target screening condition is obtained, and data screening processing is performed on the original data according to the rule combination data and the target screening condition to obtain target screening data. Compared with related technologies, the embodiment of the application can realize fast screening of the original data according to the rule combination data and the target screening condition for the target screening condition to be screened in actual application, and then more accurate target screening data is obtained. The embodiment of the application can effectively improve the data screening efficiency.

[0064] The embodiment of the application provides a data screening method, device, equipment and storage medium, which is specifically explained through the following embodiments. First, the data screening method in the embodiment of the application is described.

[0065] The embodiment of the application can acquire and process related data such as original data in the embodiment of the application based on artificial intelligence technology. The artificial intelligence is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results.

[0066] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and the like. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology and machine learning / deep learning and the like.

[0067] The data screening method provided in the embodiments of the present application relates to the technical field of artificial intelligence. The data screening method provided in the embodiments of the present application can be applied to a terminal, can also be applied to a server end, and can further be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart watch or the like; the server end can be configured as a stand-alone physical server, can also be configured as a server cluster or a distributed system formed by multiple physical servers, and can further be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application for implementing the data screening method, but is not limited to the above forms.

[0068] Specifically, the terminal / device can obtain the original data, and the terminal / device can be a mobile terminal device or a non-mobile terminal device. The mobile terminal device can be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a palm computer, an ultra-mobile personal computer (UMPC), a wearable device, a netbook, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, or the like; and the non-mobile terminal device can be a personal computer, a teller machine, or a self-service machine, and the embodiments of the present application are not limited in this regard.

[0069] The embodiments of the present application can be used in many general or special computer system environments or configurations. For example, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0070] Specifically, refer to Figure 1The embodiment of the present application provides a data screening method, including but not limited to the following steps S100 to S400:

[0071] Step S100, obtaining original data, performing data processing on the original data to obtain label data;

[0072] Step S200, creating target rule data corresponding to the label data according to the label data, wherein at least two target screening rules are stored in the target rule data;

[0073] Step S300, obtaining serial number data corresponding to each target screening rule, performing data combination processing on the serial number data to obtain rule combination data;

[0074] Step S400, obtaining a target screening condition, performing data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data.

[0075] Exemplarily, the data screening method of the embodiment of the present application can be applied to a large batch of data screening scene, and can be applied to a large batch of data screening rule editing platform.

[0076] In some embodiments of the present application, for a big data synchronization task, the original data such as detail data is periodically imported into the related data table of the ERS to serve as a data source for business statistics. After obtaining the original data, data processing is performed on the original data to obtain label data.

[0077] Exemplarily, the original data is processed to obtain label data, including but not limited to the following steps: performing data classification processing on the original data according to a preset label to obtain label data. For example, label data can be created according to preset labels of different conditions. The label data can be in the form of a label dictionary table, for example, the label data is specifically: having illegal risk behavior records in the past two years, having large amount of lending records in the past five years, and the like, and different fields in the original data (such as different detail data) can be corresponded according to the label data (such as the label dictionary table).

[0078] In some embodiments of the present application, the target rule data is created, and the target rule data can be a rule detail table. Exemplarily, a user can set a target screening rule according to a data screening requirement through a large batch of data screening rule editing platform to store the target screening rule in the target rule data. For example, a target screening rule designed according to a data screening requirement is stored in the rule detail table. It should be noted that the label data corresponds to the target rule data, that is, for a certain type of label data, there is corresponding target rule data. And at least two target screening rules are stored in the target rule data, and the data is screened through the target screening rule.

[0079] In some embodiments of the present application, the rule combination data is created, which can be a rule combination table. Specifically, the rule combination data is obtained by acquiring serial number data corresponding to each target screening rule and performing data combination processing on the serial number data. For example, the serial number data can be rule data with an ID, such as rule 1, rule 2, rule 3, and the like. The rule combination data is obtained by performing data combination processing on the rule data with an ID. For example, the target rule data can be a rule detail table, that is, the rule detail table stores at least two target screening rules. The rule combination data, which is a more complex screening condition, is obtained by performing data combination processing on the at least two target screening rules in the rule detail table using the serial number data. By setting a complex screening condition, the screening requirement of big data can be effectively met.

[0080] In some embodiments, the rule combination data can be rule 1 and rule 2, or rule 1 or rule 2. In other embodiments, the execution order of the multiple target screening rules can be defined in the rule combination table in the form of parentheses in the front-end page, that is, a parenthesis button is set in the front-end page, and the rule combination table is directly recorded as “()” in the back-end.

[0081] For example, the rule combination data can be rule 1 and (rule 2 excluding rule 3). According to the rule combination data, the data execution order is that the data of rule 3 is excluded from the data of rule 2 to obtain a screening result, and then the screening result is subjected to data aggregation processing and data deduplication processing (that is, data merging / aggregation and then deduplication) with the data of rule 1 to obtain the final target screening data.

[0082] The rule combination data of the embodiments of the present application is pre-set / stored in the large-volume data screening rule editing platform. Therefore, in actual application, the target screening condition input by the user is acquired, and then the original data is subjected to data screening processing according to the rule combination data and the target screening condition to obtain the target screening data. It should be noted that the target screening condition is for the original data. The target screening data required by the user is quickly screened from the original data by the target screening condition input by the user and according to the rule combination data, which can effectively improve the data screening efficiency.

[0083] The beneficial effects of the embodiments of the present application include: obtaining original data, performing data processing on the original data to obtain label data; creating target rule data corresponding to the label data according to the label data, wherein at least two target screening rules are stored in the target rule data; obtaining serial number data corresponding to each target screening rule, performing data combination processing on the serial number data to obtain rule combination data; obtaining a target screening condition, performing data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data. Compared with related technologies, the embodiments of the present application can realize fast screening of the original data according to the rule combination data and the target screening condition for the target screening condition to be screened in actual application, and then obtain more accurate target screening data. The embodiments of the present application can effectively improve the data screening efficiency.

[0084] Reference Figure 2 It can be understood that the target rule data is provided with at least two of the one-to-one corresponding label item, the operator item, the screening condition item, the time range item, the comparison value item and the numerical value type item. After the target rule data corresponding to the label data is created according to the label data, the following steps S210 to S270 are included but not limited to:

[0085] Step S210, obtaining a target screening rule, performing data splitting processing on the target screening rule to obtain splitting data corresponding to the target screening rule, wherein the splitting data corresponds to serial number data;

[0086] Step S220, in the case that the splitting data includes a target label, storing the target label in the label item of the target rule data;

[0087] Step S230, in the case that the splitting data includes a target operator, storing the target operator in the operator item of the target rule data;

[0088] Step S240, in the case that the splitting data includes a target screening condition, storing the target screening condition in the screening condition item of the target rule data;

[0089] Step S250, in the case that the splitting data includes a target time range, storing the target time range in the time range item of the target rule data;

[0090] Step S260, in the case that the splitting data includes a target comparison value, storing the target comparison value in the comparison value item of the target rule data;

[0091] Step S270, in the case that the splitting data includes a target numerical value type, storing the target numerical value type in the numerical value type item of the target rule data.

[0092] Exemplarily, the target rule data can be a rule detail table. Therefore, at least two of the label item, the operator item, the filtering condition item, the time range item, the comparison value item and the numerical type item are provided in the target rule data (e.g. in the rule detail table). For example, the rule detail table provides the label item and the operator item one by one; or the rule detail table provides the label item, the operator item, the filtering condition item, the time range item, the comparison value item and the numerical type item one by one, which is not limited in the embodiments of the present application.

[0093] Specifically, before storing the target filtering rule in the target rule data, the target filtering rule needs to be acquired, and then the data splitting processing is performed on the target filtering rule to obtain the splitting data corresponding to the target filtering rule.

[0094] It should be noted that the splitting data includes at least two of the target label, the target operator, the target filtering condition, the target time range, the target comparison value and the target numerical type, and the target label corresponds to the label item, the target operator corresponds to the operator item, the target filtering condition corresponds to the filtering condition item, the target time range corresponds to the time range item, the target comparison value corresponds to the comparison value item, and the target numerical type corresponds to the numerical type item; that is, after obtaining the splitting data corresponding to the target filtering rule, it is further needed to determine which data the splitting data includes to perform at least one of the steps S220 to S270.

[0095] Since the target filtering rule corresponds to the splitting data, and the target filtering rule corresponds to the serial number data, the serial number data corresponds to the splitting data, that is, one-to-one corresponds to the target label, the target operator, the target filtering condition, the target time range, the target comparison value and the target numerical type.

[0096] Exemplarily, the target filtering rule is input through the front-end page, and each target filtering rule is subjected to data splitting processing, so as to store the splitting data in the label item, the operator item, the filtering condition item, the time range item, the comparison value item or the numerical type item corresponding to the target rule data.

[0097] For example, for the target filtering rule of searching the data amount of the C label greater than 0 from the A table, the target rule data such as the rule detail table will be stored in the form of the following table 1:

[0098] Table 1

[0099]

[0100] Among them, Null represents null value.

[0101] For example, for a target filtering rule of filtering data whose value corresponding to C label is 100 larger than value corresponding to B label from A table, corresponding target rule data such as rule detail table will be stored in the form of the following table 2:

[0102] Table 2

[0103]

[0104] For example, for a target filtering rule of filtering data whose value corresponding to C label is 100 larger than value corresponding to B label from A table, corresponding target rule data such as rule detail table will be stored in the form of the following table 2:

[0105] Table 3

[0106]

[0107] It should be noted that for each target filtering rule, there is corresponding serial number data, such as rule 1, rule 2, rule 3, etc., which is not specifically limited by the present application.

[0108] Referring to Figure 3 It can be understood that the serial number data is subjected to data combination processing to obtain rule combination data, including but not limited to the following step S310:

[0109] In step S310, the serial number data is subjected to data combination processing according to a preset execution instruction to obtain rule combination data, wherein the preset execution instruction includes a preset execution sequence instruction and / or a preset execution condition instruction.

[0110] In the embodiment of the present application, the serial number data can be subjected to data combination processing according to a preset execution sequence instruction to obtain rule combination data, for example, the rule combination data can be rule 1 rule 2, that is, first, according to rule 1 to filter the original data to obtain first target filtering data, and then according to rule 2 to filter the original data to obtain second target filtering data, and output the first target filtering data and the second target filtering data.

[0111] Or, the serial number data can be subjected to data combination processing according to a preset execution condition instruction to obtain rule combination data, for example, the rule combination data can be rule 1 excluding rule 2, that is, first, according to rule 1 to filter the original data to obtain first target filtering data, and then according to rule 2 to filter the original data to obtain second target filtering data, and then the second target filtering data in the first target filtering data is removed to obtain target filtering data.

[0112] The serial number data can be subjected to data combination processing according to preset execution order instructions and preset execution condition instructions to obtain rule combination data. For example, the rule combination data can be rule 1 and (rule 2 excluding rule 3). The above examples are used to better illustrate the rule combination data of the embodiments of the present application, and do not limit the rule combination data of the embodiments of the present application.

[0113] The embodiments of the present application set the rule combination data in advance, which facilitates quick screening of the original data and effectively improves the data screening efficiency.

[0114] Referring to Figure 4 It can be understood that after obtaining the target screening condition, the following steps S410 to S420 are included but not limited thereto:

[0115] In step S410, the target screening condition is subjected to rule splitting processing according to the label data to obtain target screening rules corresponding to the target screening condition.

[0116] In step S420, the target screening rules are subjected to data combination processing to obtain rule target combination data.

[0117] It should be noted that in order to better screen the original data according to the target screening condition, the embodiments of the present application can split the target screening condition according to the label data to obtain target screening rules corresponding to the target screening condition, and then obtain serial number data corresponding to the target screening rules, and subject the serial number data corresponding to the target screening rules to data combination processing, i.e., subject the target screening rules to data combination processing to obtain rule target combination data.

[0118] For example, after obtaining the target screening condition input by the user, the target screening condition can be split into a plurality of target screening rules, and then the plurality of target screening rules are subjected to data combination processing through the processing of step S300 to obtain rule target combination data. Then, it is further determined whether the rule target combination data belongs to the preset rule combination data.

[0119] Referring to Figure 5 It can be understood that the original data is subjected to data screening processing according to the rule combination data and the target screening condition to obtain target screening data, including but not limited to the following step S430:

[0120] In step S430, when the rule combination data includes rule target combination data, the original data is subjected to data screening processing according to the rule target combination data to obtain target screening data.

[0121] In the embodiment of the present application, the rule target combination data obtained according to the target screening condition is further used to determine whether the rule target combination data belongs to the preset rule combination data. In the case where the rule combination data includes the rule target combination data, the original data is subjected to data screening processing according to the rule target combination data to obtain the target screening data. The embodiment of the present application can perform data screening processing on the original data through the stored rule combination data, thereby improving the data screening efficiency.

[0122] With reference to Figure 6 It can be understood that the original data is subjected to data screening processing according to the rule combination data and the target screening condition to obtain the target screening data, which includes but is not limited to the following steps S440 to S460:

[0123] In the case where the rule combination data does not include the rule target combination data, the rule test instruction corresponding to the rule target combination data is acquired.

[0124] In the case where the rule combination data does not include the rule target combination data, the rule test instruction corresponding to the rule target combination data is acquired.

[0125] In the case where the rule combination data does not include the rule target combination data, the rule test instruction corresponding to the rule target combination data is acquired.

[0126] In the embodiment of the present application, the rule target combination data obtained according to the target screening condition is further used to determine whether the rule target combination data belongs to the preset rule combination data. In the case where the rule combination data does not include the rule target combination data, the rule test instruction corresponding to the rule target combination data is acquired. The rule test instruction is set to provide a trial calculation function for the user, so as to test whether the rule target combination data obtained through steps S410 to S420 can realize the data screening function.

[0127] Specifically, the rule target combination data is subjected to data matching processing with the target rule data and the rule combination data according to the rule test instruction to obtain matching data. In the case where the matching data satisfies a preset matching condition, the original data is subjected to data screening processing according to the rule target combination data to obtain the target screening data.

[0128] Exemplarily, the trial function is provided on the rule writing page, after the user clicks the trial function button, the backend obtains the rule test instruction corresponding to the rule target combination data, then, data matching processing is performed according to the serial number data corresponding to the rule target combination data and the target rule data (for example, the rule combination table) and the rule combination data (for example, the rule detail table), and matching data is obtained. The trial function provided in the embodiment of the application is to facilitate the user to verify whether the rule target combination data can filter out data.

[0129] It should be noted that the rule target combination data in the embodiment of the application is rule target combination data written by the user according to the target filtering condition, which does not belong to the rule combination data preset.

[0130] Exemplarily, the preset matching condition is that when the matching data, for example, the trial result, is not the preset value, the matching data satisfies the preset matching condition. For example, when the matching data, for example, the trial result, is not 0 (preset value), it indicates that the rule target combination data meets the data filtering requirement, then, the original data is filtered according to the rule target combination data to obtain the target filtering data, that is, the target filtering data is filtered from the original data, for example, the detail data, corresponding to the label data.

[0131] When the matching data, for example, the trial result, is the preset value, the matching data does not satisfy the preset matching condition. For example, when the matching data, for example, the trial result, is 0 (preset value), it indicates that the rule target combination data does not meet the data filtering requirement, at this time, the execution steps S410 to S420 are needed to obtain the modified rule target combination data, that is, the target filtering condition is reprocessed by rule splitting to obtain the modified target filtering rule, and the modified target filtering rule is processed by data combination to obtain the modified rule target combination data.

[0132] The embodiment of the application can determine whether the rule target combination data meets the data filtering requirement through the rule test instruction, and in the case that the rule target combination data does not meet the data filtering requirement, the rule target combination data can be re-modified for trial function again, through such setting, the accuracy of data filtering can be effectively improved.

[0133] In some embodiments, when the matching data satisfies the preset matching condition, the rule target combination data satisfying the preset matching condition is stored in the rule combination data, so that the rule combination data includes the rule target combination data, thereby facilitating subsequent direct filtering of the target filtering data through step S430 when facing the same rule target combination data, and effectively improving the data filtering efficiency.

[0134] Reference Figure 7It can be understood that the data screening processing of the original data according to the rule target combination data obtains target screening data, including but not limited to the following steps S461 to S462:

[0135] Step S461, data screening processing of the original data according to the rule target combination data obtains initial screening data;

[0136] Step S462, data aggregation processing and data deduplication processing of the initial screening data obtains target screening data.

[0137] When the matching data meets the preset matching condition, that is, the matching data such as the trial result is not 0 (preset value), it indicates that the rule target combination data meets the data screening requirement, at this time, the data screening processing of the original data according to the rule target combination data obtains initial screening data, and then the data aggregation processing and data deduplication processing of the initial screening data obtains target screening data. The obtained target screening data can be recorded in the model result table of the database for the user to export or subsequent analysis.

[0138] A specific embodiment is described to describe the data screening method of the embodiment of the application:

[0139] In actual application, a target screening condition is obtained, which can be that the user needs to screen empno (employee number) of all employees in the original data, which has large loan disputes in the past five years, and whose single amount growth is 0 in the past half year, and excludes empno (employee number) whose complaint number is 0 in the past 2 years from the initial screening result.

[0140] According to the label data, the rule splitting processing of the target screening condition is performed to obtain three target screening rules corresponding to the target screening condition, and the three target screening rules are:

[0141] Rule 1: label [large loan dispute case number in the past 5 years] > 0;

[0142] Rule 2: label [single amount] - [single amount (last 6 months)] = 0;

[0143] Rule 3: label [complaint number in the past 2 years] = 0;

[0144] The serial number data corresponding to the three target screening rules (i.e., rule 1, rule 2, and rule 3) is obtained, and the data combination processing of the serial number data corresponding to the target screening rules is performed to obtain rule target combination data:

[0145] The rule target combination data is: (rule 1 and rule 2) excluding rule 3.

[0146] In the case that the rule combination data does not include the rule target combination data, rule test instructions corresponding to the rule target combination data are acquired, and then matching data is obtained. When the matching data, for example, a trial result, is not 0, it indicates that the rule target combination data meets the data screening requirement, and the rule target combination data can be saved and repeatedly called. When the matching data, for example, the trial result, is 0, it indicates that the rule target combination data does not meet the data screening requirement, and the rule target combination data is not saved.

[0147] In the embodiment of the application, when a user needs to screen data of a certain target, the user can screen data by using the rule combination data or the rule target combination data stored on the mass data screening rule editing platform, or by using a new rule target combination data written by the user to run the specified rule target combination data to screen data.

[0148] The data screening method of the embodiment of the application can significantly save the working hours consumed by business personnel, IT personnel and test personnel due to different understanding of requirements, effectively improve the utilization rate of existing screening rules, avoid repeated work, and thus effectively improve the efficiency and accuracy of data screening.

[0149] In addition, the embodiment of the application can enable the user to set commonly used data screening rules, that is, rule target combination data, according to the user's own needs, and can timely feedback the screening result, so as to reduce repeated work, improve the flexibility of mass data screening, shorten the time for screening data of a specific type, and improve the practicability.

[0150] The data screening method of the embodiment of the application comprises the following steps: obtaining original data, performing data processing on the original data to obtain label data; creating target rule data corresponding to the label data according to the label data, wherein at least two target screening rules are stored in the target rule data; obtaining serial number data corresponding to each target screening rule, performing data combination processing on the serial number data to obtain rule combination data; obtaining a target screening condition, and performing data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data. Compared with related technologies, the embodiment of the application can realize fast screening of the original data according to the rule combination data and the target screening condition for the target screening condition to be screened in actual application, and thus obtain more accurate target screening data. The embodiment of the application can effectively improve the data screening efficiency.

[0151] Reference Figure 8 An embodiment of the application further provides a data screening device, which can implement the data screening method, and the data screening device comprises but is not limited to the following modules:

[0152] The data processing module 100 is configured to acquire original data, and perform data processing on the original data to obtain label data.

[0153] The data creating module 200 is configured to create target rule data corresponding to the label data according to the label data, wherein the target rule data stores at least two target screening rules.

[0154] The data combination module 300 is configured to acquire serial number data corresponding to each target screening rule, and perform data combination processing on the serial number data to obtain rule combination data.

[0155] The data screening module 400 is configured to acquire a target screening condition, and perform data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data.

[0156] It should be noted that the contents of the method embodiments of the present application are applicable to the device embodiments, the device embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments, which will not be described in detail.

[0157] The present application also provides a data screening device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing the connection and communication between the processor and the memory, and the program is executed by the processor to realize the above data screening method. The data screening device can be any intelligent terminal such as a tablet computer and a vehicle-mounted computer.

[0158] It should be noted that the data screening device in the present embodiment can be applied to the data screening method of the above embodiments, and the data screening device in the present embodiment and the data screening method of the above embodiments have the same inventive concept, so these embodiments have the same implementation principle and technical effects, which will not be described in detail.

[0159] Please refer to Figure 9 , Figure 9 The hardware structure of the data screening device will be described in detail. The data screening device comprises a processor 801, a memory 802, an input / output interface 803, a communication interface 804 and a bus 805.

[0160] The processor 801 can be implemented by a general CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute a related program to implement the technical solutions provided by the present application.

[0161] The memory 802 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 802 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 802 and are called and executed by the processor 801 to perform the data screening method of the embodiments of the present application;

[0162] The input / output interface 803 is configured to realize information input and output.

[0163] The communication interface 804 is configured to realize the communication interaction between the device and other devices, and the communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0164] The bus 805 is configured to transmit information between various components (for example, the processor 801, the memory 802, the input / output interface 803, and the communication interface 804) of the device.

[0165] The processor 801, the memory 802, the input / output interface 803, and the communication interface 804 are connected to each other through the bus 805 to realize the communication connection between the devices.

[0166] The data screening device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, that is, can be located in one place or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to realize the purpose of the embodiments of the present application.

[0167] The embodiments of the present application also provide a computer readable storage medium for computer readable storage, and the computer readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the data screening method.

[0168] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include memory that is remotely located with respect to the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0169] The data screening method, device, equipment and storage medium of the embodiments of the present application obtain original data, perform data processing on the original data to obtain label data, create target rule data corresponding to the label data according to the label data, wherein at least two target screening rules are stored in the target rule data, obtain serial number data corresponding to each target screening rule, perform data combination processing on the serial number data to obtain rule combination data, obtain a target screening condition, and perform data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data. Compared with related technologies, the embodiments of the present application can realize fast screening of the original data according to the rule combination data and the target screening condition for the target screening condition to be screened in actual application, and then obtain more accurate target screening data. The embodiments of the present application can effectively improve the data screening efficiency.

[0170] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0171] Those skilled in the art can understand that, Figures 1-7 The technical solutions shown in the above description do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0172] The device embodiments described above are only schematic, and units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0173] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware or appropriate combination thereof.

[0174] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a general order and / or structure unless otherwise indicated. Furthermore, the terms "comprise", "comprising", "has", "having", "includes", "including", "contain", "containing" or any other similar forms are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains items or components does not include items or components not explicitly recited. The terms "a" or "an", as used herein in the detailed description and in the claims, mean "one or more" or "at least one", unless otherwise indicated.

[0175] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0176] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components above can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0178] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0179] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0180] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A method of data screening, characterized by, The method comprises the following steps: acquiring original data, and performing data processing on the original data to obtain label data; creating target rule data corresponding to the label data according to the label data, wherein at least two target screening rules are stored in the target rule data; the target rule data is provided with at least two of a label item, an operator item, a screening condition item, a time range item, a comparison value item and a numerical type item in one-to-one correspondence; acquiring serial number data corresponding to each target screening rule, and performing data combination processing on the serial number data to obtain rule combination data; acquiring a target screening condition, and performing data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data; the data combination processing on the serial number data to obtain rule combination data comprises: performing data combination processing on the serial number data according to a preset execution instruction to obtain rule combination data, wherein the preset execution instruction comprises a preset execution sequence instruction and / or a preset execution condition instruction.

2. The data screening method of claim 1, wherein, after the step of creating target rule data corresponding to the label data according to the label data, the method comprises the following steps: acquiring a target screening rule, and performing data splitting processing on the target screening rule to obtain splitting data corresponding to the target screening rule, wherein the splitting data corresponds to the serial number data; in the case that the splitting data comprises a target label, the target label is stored in the label item of the target rule data; in the case that the splitting data comprises a target operator, the target operator is stored in the operator item of the target rule data; in the case that the splitting data comprises a target screening condition, the target screening condition is stored in the screening condition item of the target rule data; in the case that the splitting data comprises a target time range, the target time range is stored in the time range item of the target rule data; in the case that the splitting data comprises a target comparison value, the target comparison value is stored in the comparison value item of the target rule data; in the case that the splitting data comprises a target numerical type, the target numerical type is stored in the numerical type item of the target rule data.

3. The data screening method according to any one of claims 1 to 2, characterized in that, after acquiring a target screening condition, the method comprises the following steps: performing rule splitting processing on the target screening condition according to the label data to obtain a target to-be-screened rule corresponding to the target screening condition; performing the data combination processing on the target to-be-screened rule to obtain rule target combination data.

4. The data screening method of claim 3, wherein, the data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data comprises: in the case that the rule combination data comprises the rule target combination data, performing data screening processing on the original data according to the rule target combination data to obtain target screening data.

5. The data screening method of claim 3, wherein, the data screening processing on the original data according to the rule combination data and the target screening condition to obtain target screening data comprises: In a case where the rule combination data does not include the rule target combination data, acquiring rule test instructions corresponding to the rule target combination data; According to the rule test instructions, performing data matching processing on the rule target combination data, the target rule data and the rule combination data to obtain matching data; When the matching data meets a preset matching condition, performing data screening processing on the original data according to the rule target combination data to obtain target screening data.

6. The data screening method of claim 5, wherein, The data screening processing on the original data according to the rule target combination data to obtain target screening data comprises: Performing data screening processing on the original data according to the rule target combination data to obtain initial screening data; Performing data aggregation processing and data deduplication processing on the initial screening data to obtain the target screening data.

7. A data screening device characterized by comprising: Comprise: The data processing module is used for acquiring original data, performing data processing on the original data, and obtaining label data; The data creation module is used for creating target rule data corresponding to the label data according to the label data, wherein the target rule data stores at least two target screening rules; The data combination module is used for acquiring serial number data corresponding to each target screening rule, performing data combination processing on the serial number data, and obtaining rule combination data; The data screening module is used for acquiring a target screening condition, performing data screening processing on the original data according to the rule combination data and the target screening condition, and obtaining target screening data; the target rule data is provided with at least two of a label item, an operator item, a screening condition item, a time range item, a comparison value item and a numerical type item; The data combination processing on the serial number data to obtain rule combination data comprises: According to a preset execution instruction, the serial number data is subjected to data combination processing to obtain rule combination data, wherein the preset execution instruction comprises a preset execution sequence instruction and / or a preset execution condition instruction.

8. A data screening device characterized by, Comprise: A memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data screening method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer executable program is stored, and the computer executable program is used to execute the data screening method according to any one of claims 1 to 6.

Images