Data processing method, device, equipment, storage medium and computer program product

Abstract

The present application discloses a data processing method, apparatus, device, storage medium, and computer program product. The method comprises: obtaining unprocessed educational data within a preset area; the unprocessed educational data includes multiple different types of educational data; performing a hash function calculation on the unprocessed educational data to obtain a hash code corresponding to the unprocessed educational data; constructing a hash index based on the hash code; scanning the hash index to obtain at least one frequent item set; wherein the frequent item set includes at least one type of educational data; and generating association rules for each of the frequent item sets based on the at least one frequent item set to efficiently discover the associations between different types of educational data from massive amounts of educational data.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method, apparatus, device, storage medium, and computer program product. Background Art

[0002] With the advent of the big data era, the processing of massive data has become a key issue to be solved. Especially as people attach great importance to the education industry, discovering the correlation between massive educational data has become an even more important issue to be solved.

[0003] Currently, the correlation between massive educational data is usually discovered by using the Aprior algorithm to repeatedly scan the large amount of educational data to be processed layer by layer, so as to discover the correlation between various types of educational data in the massive educational data.

[0004] However, the above method requires repeated scanning of the database storing massive educational data layer by layer, which increases the number of operations to read the database I / O port, resulting in excessively long calculation time and reduced data processing efficiency. Summary of the Invention

[0005] The embodiments of the present application provide a data processing method, apparatus, device, storage medium and computer program product to achieve the effect of efficiently discovering the correlation between various types of educational data from massive educational data.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0007] Performing hash function calculation on the education data to be processed to obtain a hash code corresponding to the education data to be processed;

[0008] Constructing a hash index for the hash code;

[0009] Scanning the hash index to obtain at least one frequent item set; wherein the frequent item set includes at least one type of educational data;

[0010] Based on at least one frequent item set, association rules of the frequent item sets are generated.

[0011] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0012] An acquisition module, configured to acquire the education data to be processed within a preset area; the education data to be processed includes a plurality of different types of education data;

[0013] a calculation module, configured to calculate a hash function on the education data to be processed to obtain a hash code corresponding to the education data to be processed;

[0014] A construction module, configured to construct a hash index for the hash code;

[0015] A scanning module, configured to scan the hash index to obtain at least one frequent item set; wherein the frequent item set includes at least one type of educational data;

[0016] The association rule generating module is configured to generate association rules for each frequent item set based on at least one frequent item set.

[0017] In a third aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0018] a processor and a memory storing computer program instructions;

[0019] When the processor executes the computer program instructions, it implements the data processing method provided in the embodiment of the first aspect of the present application.

[0020] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the data processing method provided in the embodiment of the first aspect of the present application is implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data processing method provided in the embodiment of the first aspect of the present application.

[0022] The data processing method, apparatus, device, storage medium and computer program product of the embodiments of the present application calculate a hash function on the acquired educational data to be processed to obtain a hash code corresponding to the educational data to be processed, then construct a hash index for the hash code, and then scan the hash index to generate at least one frequent item set, and then based on the at least one frequent item set, association rules for each frequent item set can be generated. Since the educational data to be processed includes multiple different types of educational data, and the frequent item set includes at least one type of educational data, the number of scans of the I / O port of the database is reduced by scanning the hash index instead of scanning the acquired educational data to be processed, thereby improving the scanning efficiency, thereby improving the processing efficiency of the educational data to be processed, and efficiently obtaining the association relationship between various types of educational data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 is a flowchart of a data processing method provided by an embodiment of the first aspect of the present application;

[0025] Figure 2 Schematic diagram of the hash index structure involved in the embodiment of the first aspect of the present application;

[0026] Figure 3 is a flowchart of the Apriori association mining method involved in the embodiment of the first aspect of the present application;

[0027] Figure 4 1 is a schematic diagram of a framework for implementing a data processing method according to an embodiment of the first aspect of the present application;

[0028] Figure 5 is a structural diagram of a data processing device provided by an embodiment of the third aspect of the present application;

[0029] Figure 6 It is a structural diagram of the data processing device provided in an embodiment of the fourth aspect of the present application. DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0031] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0032] As described in the background technology section, the prior art uses the Aprior algorithm to repeatedly scan a large amount of educational data to be processed layer by layer, thereby discovering the correlation between various types of educational data in the massive educational data. This method increases the number of operations to read the database I / O port, resulting in excessive calculation time and reduced data processing efficiency. In order to solve the above problems, the embodiments of the present application provide a data processing method, device, equipment, storage medium and computer program product, which calculates a hash function on the acquired educational data to be processed to obtain a hash code corresponding to the educational data to be processed, then constructs a hash index for the hash code, and then scans the hash index to generate at least one frequent item set. Based on the at least one frequent item set, association rules for each frequent item set can be generated. Since the educational data to be processed includes multiple different types of educational data, and the frequent item set includes at least one type of educational data, the number of scans of the database I / O port is reduced by scanning the hash index instead of scanning the acquired educational data to be processed, thereby improving the scanning efficiency and thus improving the processing efficiency of the educational data to be processed, and efficiently obtaining the correlation between various types of educational data.

[0033] The data processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0034] Figure 1 FIG. 1 shows a flow chart of a data processing method provided by an embodiment of the present application. Figure 1 As shown, the data processing method provided in the embodiment of the present application includes the following steps:

[0035] Step 110: Obtain the education data to be processed within the preset area.

[0036] The educational data to be processed may include a variety of different types of educational data.

[0037] Step 120: Calculate a hash function on the education data to be processed to obtain a hash code corresponding to the education data to be processed.

[0038] Step 130: Build a hash index for the hash code.

[0039] Step 140: Scan the hash index to obtain at least one frequent itemset.

[0040] The frequent item set may include at least one type of educational data.

[0041] Step 150: Generate association rules for each frequent item set based on at least one frequent item set.

[0042] In an embodiment of the present application, a hash function is calculated on the acquired educational data to be processed to obtain a hash code corresponding to the educational data to be processed, and then a hash index is constructed for the hash code, and then the hash index is scanned to generate at least one frequent item set, and then based on at least one frequent item set, association rules for each frequent item set can be generated. Since the educational data to be processed includes many different types of educational data, and the frequent item set includes at least one type of educational data, the number of scans of the I / O port of the database is reduced by scanning the hash index instead of scanning the acquired educational data to be processed, thereby improving the scanning efficiency, and thus improving the processing efficiency of the educational data to be processed, and efficiently obtaining the association relationship between various types of educational data.

[0043] The data processing method mentioned above is described in detail below.

[0044] First, step 110 is introduced to obtain the education data to be processed within the preset area.

[0045] The preset area may be a pre-set area, specifically an area where education data analysis is to be performed in advance, such as a provincial area or a municipal area, etc. This is not limited here.

[0046] The educational data to be processed may be educational data to be processed.

[0047] In some embodiments of the present application, the education data to be processed may include multiple different types of education data, for example, at least two of the following: student growth data, teacher development data, class management data, and school operation data.

[0048] In some embodiments of the present application, the education data to be processed may be obtained from a database storing education data to be processed in a preset area.

[0049] In some embodiments of the present application, student growth data may mainly include student information, subject performance information, attendance information from school to school, moral education evaluation information, and personal commendation information.

[0050] In some embodiments of the present application, teacher development data may mainly include teacher information, teaching evaluation information, academic research information, commendation type information and attendance management information, etc.

[0051] In some embodiments of the present application, class management data may mainly include class information, teacher evaluation information, subject analysis information, class commendation information, etc.

[0052] In some embodiments of the present application, school operation data may mainly include school information, teaching and research activity information, regional joint examination information, school commendation information, asset management information, and safety management information, etc.

[0053] Then, step 120 is introduced, in which a hash function is calculated on the education data to be processed to obtain a hash code corresponding to the education data to be processed.

[0054] In some embodiments of the present application, the hash function calculation of the educational data to be processed can be performed using the binary hash function in the following formula (1) to calculate the educational data to be processed, and obtain the binary hash code corresponding to the educational data to be processed, that is, obtain the m-bit binary hash code G(x) corresponding to the educational data to be processed.

[0055]

[0056] Among them, x is the education data to be processed; ω T Represents vector transpose.

[0057] Then, step 130 is introduced to construct a hash index for the hash code.

[0058] In some embodiments of the present application, in order to further achieve the effect of efficiently discovering the correlation between various types of educational data from massive educational data, step 130 may specifically include:

[0059] Dividing the hash code into at least two sub-hash code strings;

[0060] Each sub-hash code string is encoded as a hash barrel, inserted into the hash table, and a hash index is generated.

[0061] The sub-hash code string may be a code string formed by dividing the hash code.

[0062] In some embodiments of the present application, a parameter L may be selected to divide the m-bit Hamming space into L s-bit sub-Hamming spaces, and a corresponding hash index table {T1, T2, ... T L Secondly, the hash code G(x) of each transaction data object is divided into L unconnected sub-code strings {G(x)1, G(x)2, ... G(x) L}, and use each substring as a hash bucket number, and finally insert the education data to be processed into the corresponding hash table to form a hash index.

[0063] In an example, taking four educational data to be processed x1, x2, x3 and x4 as examples, after being encoded by an 8-bit binary hash function, the hash codes obtained are G(x1)=00011011, G(x2)=10110100, G(x3)=00101111, G(x4)=10000001. Then, each hash code is divided into four 2-bit sub-hash code strings, and then the four 2-bit sub-hash code strings are used as hash barrel codes. For example, the hash barrel numbers of x1 in hash tables T1, T2, T3 and T4 are "00", "01", "10" and "11" respectively. Finally, the corresponding educational data to be processed x are inserted into the corresponding hash tables to obtain the hash index. The binary hash index structure is as follows: Figure 2 shown.

[0064] In an embodiment of the present application, by dividing the hash code into at least two sub-hash code strings; each sub-hash code string is used as a hash barrel code and inserted into a hash table to generate a hash index. In this way, the hash index can be accurately generated, and then the hash index can be scanned without scanning the database I / O port, thereby further achieving the effect of efficiently discovering the correlation between various types of educational data from massive educational data.

[0065] Next, step 140 is introduced, in which the hash index is scanned to obtain at least one frequent itemset.

[0066] In some embodiments of the present application, a binary hash index algorithm may be used to complete a traversal scan of a hash index and generate at least one frequent itemset.

[0067] In some embodiments of the present application, the implementation of steps 140-150 can be based on the Apriori association mining method. The specific flowchart of the Apriori association mining method can be referred to as follows: Figure 3 Execute as shown.

[0068] In some embodiments of the present application, when a user requests to calculate association rules between various types of educational data, the Apriori algorithm is activated, and a hash index scan begins to obtain at least one frequent itemset, wherein each frequent itemset may include at least one type of educational data.

[0069] In some embodiments of the present application, in order to accurately obtain at least one frequent itemset, step 140 may specifically include:

[0070] Scan the hash index and construct a candidate K-itemset hash table; where the initial value of K is 1;

[0071] Get K-frequent itemsets based on the candidate K-itemset hash table and the preset minimum support;

[0072] Perform AND operations on each pair of K-frequent item sets to obtain K+1-frequent item sets;

[0073] The K value is updated in the manner of K=K+1, and the execution returns to perform AND operations on each pair of the K-frequent item sets to obtain K+1-frequent item sets until no new frequent item sets are generated.

[0074] The candidate K-itemset hash table may be a candidate frequent item set obtained after scanning the hash index.

[0075] In some embodiments of the present application, the minimum support and the minimum confidence (i.e. Figure 3 When K=1, the support of the 1-frequent item set is compared with the preset minimum support, and the confidence of the 1-frequent item set is compared with the preset minimum confidence (i.e. Figure 3 S2 in ), when the support of 1-frequent item set is greater than or equal to the preset minimum support, and the confidence of 1-frequent item set is greater than or equal to the preset minimum confidence, the hash index is scanned again (i.e. Figure 3 S3 in ), construct K+1-frequent item sets (i.e. Figure 3 S4 in ).

[0076] In some embodiments of the present application, the similarity measurement function between the hash codes corresponding to the two educational data to be processed (i.e., formula (2)) can be used to complete the construction of the candidate K-item set hash table, and then compared with the minimum support to obtain the K-frequent item set.

[0077] Count m (G(x1),G(x2))=m-||G(x1)-G(x2)|| H (2)

[0078] Among them, G(x1) is the hash code corresponding to a certain educational data to be processed; G(x2) is the hash code corresponding to another educational data to be processed; ||.|| H indicates the Hamming distance (the smaller the Hamming distance, the higher the similarity between the two); m is the number of bits of the hash code obtained after the hash function calculation is performed on the educational data to be processed.

[0079] Then perform AND operation on each K-frequent item set to obtain K+1-frequent item sets;

[0080] Update the K value in the manner of K=K+1, return to execute and operation on each pair of the K-frequent item sets to obtain K+1-frequent item sets, and execute to generate no new frequent item sets.

[0081] The following is a specific example to illustrate the specific implementation process of generating K+1-frequent itemsets from K-frequent itemsets:

[0082] In one example, the school operation data of multiple schools can be evaluated to determine whether the moral education, mathematics and Chinese performance of each school are excellent. The hash indexes corresponding to the moral education, mathematics and Chinese performance of each school are scanned (i.e. Figure 3 S3 in ), use the similarity measurement function between hash codes to calculate the moral education scores, mathematics scores and Chinese scores of different schools, and complete the construction of the candidate K-item set hash table (i.e. Figure 3 In S4), it is then compared with the minimum support. Specifically, the unprocessed educational data with a support greater than the minimum support are selected and processed to obtain 1-frequent item sets, as shown in Table 1 below.

[0083] Table 1 1-Frequent itemsets

[0084] Project Portfolio school set Binary encoding Support number I1 S1 00011011 4 I2 S2 10110100 4 I3 S3 00101111 5 I4 S4 10000001 2 … … …

[0085] In Table 1 above, the item set can be interpreted as the number of schools in the set. Here, I1 can be interpreted as the item "moral education is excellent," S1 is the set of schools where "moral education is excellent," and the support count is the number of schools in the set, such as schools 1, 2, 3, and 4. I2 is interpreted as the other items where "Chinese is excellent" or "mathematics is excellent," and the remaining school sets and support counts are similar.

[0086] The binary code in Table 1 represents "moral education is excellent". When scanning, we actually scan whether this binary code appears in the school, and then count the number of times it appears as the support number, and the schools where it appears as the school set.

[0087] In some embodiments of the present application, after generating 1-frequent item sets, each 1-frequent item set can be formed into a subsequent set C k(Right now Figure 3 S6 in , and then for the subsequent set C k (i.e., for each 1-frequent item set) the connection step is calculated (i.e. Figure 3 S7 in the previous section). This determines whether 1-frequent itemsets can be connected. If two 1-frequent itemsets meet the condition that they differ in only one item and are all the same, and are frequent itemsets, a self-connection is performed to obtain a candidate 2-itemset hash table. The hash index is traversed again, and the count is compared with the minimum support to obtain the 2-frequent itemsets that meet the conditions.

[0088] The calculation formula for the support in the association rule is shown in the following formula (3):

[0089]

[0090] Where A and B can represent two different item sets, and formula (2) is used to express the probability of A and B appearing at the same time.

[0091] In the process of generating the candidate 2-itemset hash table above, the condition that two 1-frequent itemsets differ in only one item while the rest of the items are the same and are frequent itemsets means that the two 1-frequent itemsets differ in only one item while the rest of the items are the same and are frequent itemsets. For example, given the following 1-frequent itemsets: {I1}, {I3}, and {I2}, the 1-frequent itemset {I1} and the 1-frequent itemset {I3} differ in only one item (i.e., I1 and I3 are different), and both satisfy the condition of being frequent itemsets, thus forming the candidate 2-itemset hash table {I1, I3}.

[0092] In some embodiments of the present application, after generating a 2-frequent item set, the pruning step calculation (i.e. Figure 3 Specifically, after the calculation of the connection step is performed on each 2-frequent item set and pruning is completed, the hash index is traversed and counted, and the 3-frequent item set is obtained by comparing with the minimum support.

[0093] In the above process of generating candidate 3-itemset hash tables, the condition that two 2-frequent itemsets differ in only one item and are the same as the others and are frequent itemsets is satisfied. This means that two 2-frequent itemsets differ in only one item and are the same as the others and are frequent itemsets. For example, there are the following 2-frequent itemsets: {I1,I2}, {I1,I3}, and {I1,I4}. Then, in the 2-frequent itemset {I1,I2} and the 2-frequent itemset {I1,I3}, only one item differs (i.e., I2 and I3 are different), and the other items are the same (i.e., I1 is the same), and they meet the condition of frequent itemsets. Therefore, they can form a candidate 3-itemset hash table {I1,I2,I3}, but cannot form a candidate 3-itemset hash table {I1,I1,I3}.

[0094] In the above, the 2-frequent itemset {I1,I2} and the 2-frequent itemset {I1,I3} are combined to form {I1,I2,I1,I3}. Since it is not a frequent itemset (because the elements in the frequent itemset are different), {I1,I2,I1,I3} is pruned, that is, the same elements are pruned to form {I1,I2,I3}. Therefore, the final candidate 3-itemset hash table is {I1,I2,I3}, not {I1,I1,I3}.

[0095] According to this layer-by-layer iteration, K+1-frequent item sets are generated from K-frequent item sets (that is, K is updated in the manner of K=K+1, i.e. Figure 3 S9 in ), as shown in Table 2, until higher dimensional frequent item sets cannot be generated, the loop ends.

[0096] Table 2K+1-frequent itemsets

[0097] Project Portfolio school set Binary encoding Support number <![CDATA[{I1,I2,I3,I4,…}]]> <![CDATA[{S1,S2,S3,S4,…}]]> 00101111 5

[0098] In some embodiments of the present application, using the Apriori algorithm to generate frequent itemsets is a prior art and will not be described in detail here.

[0099] It should be noted that in the process of generating K-frequent itemsets, each time a K-frequent itemset is generated, it is necessary to determine whether the K-frequent itemset is empty (i.e. Figure 3 If S5 is empty, the process ends.

[0100] In an embodiment of the present application, a hash index is scanned to construct a candidate K-itemset hash table; K-frequent itemsets are obtained based on the candidate K-itemset hash table and the minimum support; each K-frequent item set is ANDed pairwise to obtain a K+1-frequent item set; the K value is updated according to K=K+1, and the execution returns to perform the ANDing operation on each K-frequent item set pairwise to obtain a K+1-frequent item set, without generating new frequent itemsets. In this way, at least one frequent item set can be accurately obtained.

[0101] Finally, step 150 is introduced, which generates association rules for each frequent item set based on at least one frequent item set.

[0102] In some embodiments of the present application, the calculation formula for the confidence in the association rule is shown in the following formula (4):

[0103]

[0104] Among them, confidence represents the ratio of the probability of A and B appearing at the same time to the probability of A appearing. A and B can represent two different item sets.

[0105] In some embodiments of the present application, the process of combining the K-frequent itemsets to generate a K+1-frequent itemset in step 140 is the process of determining the association rule between the two K-frequent itemsets, that is, when combining the two K-frequent itemsets to generate a K+1-frequent itemset, the association rule between the two K-frequent itemsets is also determined.

[0106] In some embodiments of the present application, in order to analyze the educational data within a preset area, after step 150, the above-mentioned data processing method may further include:

[0107] Based on the association rules and the education data to be processed, the items corresponding to the education data to be processed that meet the association rules are screened out.

[0108] In some embodiments of the present application, after the association rules are known, items corresponding to the education data to be processed that meet the association rules can be screened out based on the association rules and the education data to be processed.

[0109] In an example, the association rules of the frequent item sets corresponding to the student growth data of schools A and B are known. According to the association rules (if the association rules are moral education scores) and the student growth data of schools A and B, the moral education scores (i.e., items) of the students of schools A and B can be filtered out.

[0110] In an embodiment of the present application, based on association rules and the education data to be processed, the items corresponding to the education data to be processed that meet the association rules are screened out. In this way, the items corresponding to the education data to be processed that meet the association rules can be accurately obtained, which is conducive to analyzing the items corresponding to the education data to be processed in the preset area.

[0111] In some embodiments of the present application, in order to improve the observability of the education data to be processed, after screening out items corresponding to the education data to be processed that meet the association rules based on the association rules and the education data to be processed, the data processing method may further include:

[0112] Display the project based on the preset display format.

[0113] The preset display formats may include at least one of the following: a histogram, a pie chart, a curve chart or a statistical report.

[0114] Continuing with the above example, after the moral education scores (i.e., items) of the students of schools A and B are screened out, the moral education scores of the students of schools A and B can be displayed in the form of a histogram, a pie chart, a curve chart, or a statistical report.

[0115] In the embodiments of the present application, the projects are displayed in a preset display format, so that users can view the projects intuitively, thereby improving the user experience.

[0116] In some embodiments of the present application, in order to more intuitively understand the data processing method provided by the present application, Figure 4 The framework implements the data processing method provided in this application.

[0117] Based on the data processing method provided in the above embodiment, accordingly, the embodiment of the present application also provides a specific implementation method of a data processing device, please refer to the following embodiment.

[0118] See Figure 5 , an embodiment of the present application provides a data processing device, the device comprising:

[0119] An acquisition module 510 is configured to acquire the education data to be processed within a preset area; the education data to be processed includes a variety of different types of education data;

[0120] A calculation module 520 is configured to calculate a hash function on the education data to be processed to obtain a hash code corresponding to the education data to be processed;

[0121] A construction module 530 is configured to construct a hash index for the hash code;

[0122] A scanning module 540 is configured to scan the hash index to obtain at least one frequent item set; wherein the frequent item set includes at least one type of educational data;

[0123] The association rule generating module 550 is configured to generate association rules for each of the frequent itemsets based on at least one frequent itemset.

[0124] In an embodiment of the present application, a hash function is calculated on the educational data to be processed obtained by the acquisition module through the calculation module to obtain a hash code corresponding to the educational data to be processed, and then a hash index is constructed for the hash code based on the construction module, and then the hash index is scanned based on the scanning module to generate at least one frequent item set, and then the association rule generation module can generate association rules for each frequent item set based on at least one frequent item set. Since the educational data to be processed includes many different types of educational data, and the frequent item set includes at least one type of educational data, the number of scans of the I / O port of the database is reduced by scanning the hash index instead of scanning the acquired educational data to be processed, thereby improving the scanning efficiency, thereby improving the processing efficiency of the educational data to be processed, and efficiently obtaining the association relationship between various types of educational data.

[0125] In some embodiments of the present application, the construction module 530 may specifically include:

[0126] a division unit, configured to divide the hash code into at least two sub-hash code strings;

[0127] The hash index generating unit is used to insert each of the sub-hash code strings as a hash cartridge code into the hash table to generate a hash index.

[0128] In some embodiments of the present application, the scanning module 540 may be specifically used to:

[0129] Scan the hash index to construct a candidate K-itemset hash table; wherein the initial value of K is 1;

[0130] Obtaining K-frequent itemsets according to the candidate K-itemset hash table and a preset minimum support;

[0131] Perform AND operations on each of the K-frequent item sets to obtain K+1-frequent item sets;

[0132] The K value is updated in the manner of K=K+1, and the process of performing AND operations on each pair of the K-frequent itemsets is returned to obtain K+1-frequent itemsets until no new frequent itemsets are generated.

[0133] In some embodiments of the present application, the information processing device mentioned above may further include:

[0134] The screening module is used to screen out items corresponding to the education data to be processed that meet the association rules based on the association rules and the education data to be processed.

[0135] In some embodiments of the present application, the information processing device mentioned above may further include:

[0136] A display module is used to display the project based on a preset display format; wherein the preset display format includes at least one of the following: a histogram, a pie chart, a curve chart or a statistical report.

[0137] In some embodiments of the present application, the educational data to be processed includes at least two of the following: student growth data, teacher development data, class management data, and school operation data.

[0138] Figure 6 A schematic diagram of the hardware structure of a data processing device provided in an embodiment of the present application is shown.

[0139] The data processing device may include a processor 601 and a memory 602 storing computer program instructions.

[0140] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0141] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 602 may include removable or non-removable (or fixed) media. Where appropriate, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0142] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0143] The processor 601 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .

[0144] In one example, the xx device may further include a communication interface 603 and a bus 610. Figure 3 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.

[0145] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0146] Bus 610 includes hardware, software or both, and the parts of online data flow metering equipment are coupled to each other. For example, but not limitation, bus can include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 610 can include one or more buses. Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0147] In addition, in conjunction with the data processing methods in the above embodiments, embodiments of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.

[0148] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0149] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0150] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0151] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0152] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Obtaining the education data to be processed within a preset area; The education data to be processed includes multiple different types of education data, including at least two types of student growth data, teacher development data, class management data and school operation data; Performing hash function calculation on the education data to be processed to obtain a hash code corresponding to the education data to be processed; Constructing a hash index for the hash code; Scanning the hash index to obtain at least one frequent item set; wherein the frequent item set includes at least one type of educational data; Based on at least one frequent item set, generating association rules for each of the frequent item sets; The step of constructing a hash index for the hash code includes: Dividing the hash code into at least two sub-hash code strings; Each of the sub-hash code strings is used as a hash bucket code and inserted into the hash table to generate a hash index.

2. The method according to claim 1, characterized in that Scanning the hash index to obtain at least one frequent itemset includes: Scan the hash index to construct a candidate K-itemset hash table; wherein the initial value of K is 1; Obtaining K-frequent itemsets according to the candidate K-itemset hash table and a preset minimum support; Perform AND operations on each of the K-frequent item sets to obtain K+1-frequent item sets; The K value is updated in the manner of K=K+1, and the process of performing AND operations on each pair of the K-frequent itemsets is returned to obtain K+1-frequent itemsets until no new frequent itemsets are generated.

3. The method according to claim 1, characterized in that After generating association rules of the frequent itemsets based on at least one frequent itemset, the method further includes: Based on the association rules and the education data to be processed, items corresponding to the education data to be processed that meet the association rules are screened out.

4. The method according to claim 3, characterized in that After selecting items corresponding to the education data to be processed that meet the association rules based on the association rules and the education data to be processed, the method further includes: Displaying the project based on a preset display format; The preset display form includes at least one of the following: a histogram, a pie chart, a curve chart or a statistical report.

5. The method according to any one of claims 1 to 4, characterized in that: The educational data to be processed includes at least the following two items: student growth data, teacher development data, class management data and school operation data.

6. A data processing device, characterized in that: The device comprises: An acquisition module is configured to acquire unprocessed educational data within a preset area; the unprocessed educational data includes multiple different types of educational data, including at least two types of student growth data, teacher development data, class management data, and school operation data; a calculation module, configured to calculate a hash function on the education data to be processed to obtain a hash code corresponding to the education data to be processed; A construction module, configured to construct a hash index for the hash code; A scanning module, configured to scan the hash index to obtain at least one frequent item set; wherein the frequent item set includes at least one type of educational data; An association rule generating module, configured to generate association rules for each of the frequent itemsets based on at least one frequent itemset, wherein the building module includes a partitioning unit and a hash index generating unit; The division unit is used to divide the hash code into at least two sub-hash code strings; The hash index generating unit is used to insert each of the sub-hash code strings as a hash bucket code into a hash table to generate a hash index.

7. A data processing device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data processing method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data processing method according to any one of claims 1 to 6.

Images