Method and system for increasing personal question bank generation speed

Abstract

The invention provides a method and system for increasing the generation speed of a personal question bank, and the method and system are suitable for generating the personal question bank of a tree structure, and the method comprises the steps: monitoring the number of remaining available sets of preprocessing data at set time intervals, and executing a preprocessing process when the number of the remaining available sets of preprocessing data is smaller than the set number; the preprocessing process comprises the following steps of: copying a chapter tree of an official question bank into preprocessing data, and reserving a space for identifying an information item of a user identity or setting the information item as a specific character. The method has the advantages that a whole set of data preprocessing mechanism (including the timed monitoring script and the preprocessing program) is established, so that the time consumption of program response is greatly reduced, and the generation time of the personal question bank is shortened; through a multi-process concurrent processing mechanism, the single processing efficiency of the chapter tree is improved, and the efficiency of the personal question bank preprocessing process is further improved.

Description

Technical Field

[0001] The present application belongs to the field of online education, and specifically relates to a method and system for improving the speed of generating a personal question bank. Background Art

[0002] In various online education question bank applications, in order to provide users with a personalized experience, there are usually two categories of question banks: official question banks and personal question banks. Figure 1 As shown. The official question bank refers to the content of textbook chapter data for each subject and stage, which is generally maintained by the operator, for example: "Primary School Chinese - People's Education New Curriculum Standard - Grade 3 - Textbook Chapter Tree". The personal question bank refers to individual users who use the system. They can select existing textbook chapter data from the official question bank to form their own personal question bank, so that they can organize the structure of their personal question bank according to their needs.

[0003] Normally, when you enter your personal question bank for the first time, you need to select existing textbook chapter data from the official question bank as the initialization data for your personal question bank chapter tree, and then you can make personalized adjustments based on this.

[0004] At present, there are many subjects and stages in subject education (such as "primary school mathematics", "junior high school Chinese", etc.), and the number of textbook trees contained in each subject and stage is as many as 200+. The data diagram is as follows: Figure 2 shown.

[0005] The process of initializing data from the official question bank to the personal question bank, taking the subject section of "Primary School Mathematics" as an example, the data is copied from the "Official Question Bank-Chapter Tree" to the "Personal Question Bank-Chapter Tree". The main processing flow of the program is as follows:

[0006] Step 1: Batch query the official question bank for the related data of "Textbook Version-Grade and Semester" under this subject period;

[0007] Step 2: Batch query the associated chapter tree of each "grade and semester" under the official question bank;

[0008] Step 3: Insert the "Textbook Version-Grade and Semester" data under the personal question bank;

[0009] Step 4: Based on step 3, insert the chapter tree associated with each "grade and semester" under the personal question bank;

[0010] Step 5: Loop through steps 3 and 4 until completed.

[0011] The current implementation can copy data from the "official question bank - chapter tree" to the "personal question bank - chapter tree" according to the subject section selected by the individual user (such as "primary school mathematics"). However, the current implementation has the following two defects:

[0012] 1. Tree structure data storage has dependencies;

[0013] When tree-structured data is stored in a database, it is necessary to generate and save the primary keys of the parent nodes one by one in the order of "root node-child node-leaf node" to achieve associative storage of data.

[0014] 2. The amount of data to be processed is large and the performance is low;

[0015] Taking the subject of "Primary School Mathematics" as an example, there are about 200+ "Textbook Version-Grade and Semester" associated with the subject of "Primary School Mathematics" in batch query. There are many associated chapter trees, and multiple loops are required to complete the copy. According to the calculation of 200 official chapter trees involved in "Primary School Mathematics", the processing time of each tree is calculated as 500ms, and the total time required is: 200*0.5=100 (seconds). The use of existing technical solutions will lead to obvious poor performance problems in generating personal question banks, which seriously affects the user experience. Summary of the invention

[0016] The purpose of this application is to overcome the defect of poor performance of generating personal question banks in the prior art.

[0017] In order to achieve the above purpose, the present application proposes a method for improving the speed of generating a personal question bank, which is suitable for generating a personal question bank with a tree structure. The method includes:

[0018] The remaining available number of pre-processed data is monitored at set intervals, and when the remaining available number of pre-processed data is less than the set number, the pre-processing process is executed;

[0019] The preprocessing process is: copying the chapter tree of the official question bank into the preprocessing data, and leaving the information item identifying the user identity blank or setting it to a specific character.

[0020] As an improvement of the above method, the method further comprises:

[0021] When a user selects a chapter tree from the official question bank and copies it to a personal question bank, the information item identifying the user's identity in the data of the chapter tree in the preprocessed data is directly replaced with the user's identity information.

[0022] As an improvement of the above method, the method further comprises:

[0023] After completing the replacement of the information item identifying the user identity of the chapter tree in the pre-processed data, the number of remaining available sets of the pre-processed data of the chapter tree is reduced by 1.

[0024] As an improvement of the above method, the method further comprises:

[0025] Multi-process technology is used to perform the preprocessing process.

[0026] As an improvement of the above method, when the multi-process technology is used to perform the preprocessing process, the data of each chapter tree is copied using an independent process.

[0027] As an improvement of the above method, the set time is 5 minutes.

[0028] As an improvement to the above method, the set number of sets is 10.

[0029] The present application also provides a system for improving the speed of generating a personal question bank, which is implemented based on the above method, and the system includes:

[0030] A monitoring module, used to monitor the remaining available number of pre-processed data at set intervals;

[0031] The preprocessing module is used to copy the official question bank into the preprocessing data when the remaining available number of preprocessing data is less than the set number of sets, and leave the information item identifying the user identity blank or set it to a specific character.

[0032] As an improvement of the above system, the system further comprises:

[0033] The parallel processing module is used to execute the preprocessing module using multi-process technology.

[0034] Compared with the prior art, the advantages of this application are:

[0035] 1. This application greatly reduces the program response time and improves the time for generating personal question banks by establishing a complete set of data preprocessing mechanisms (regular monitoring script + preprocessing program);

[0036] 2. This application improves the single processing efficiency of the chapter tree through a multi-process concurrent processing mechanism, and further improves the efficiency of the personal question bank preprocessing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure shows the data results of the official question bank and personal question bank in the question bank center;

[0038] Figure 2 The figure shows a schematic diagram of tree structure data in the question bank;

[0039] Figure 3 The figure shows a schematic diagram of the execution process of the method for improving the speed of generating a personal question bank;

[0040] Figure 4 The figure shows a schematic diagram of the data preprocessing process using multiple processes. DETAILED DESCRIPTION

[0041] The technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0042] Both the official question bank and the personal question bank have a tree-structured association structure. In order to store tree-structured data, it is necessary to record the serial relationship of the tree data links. This is generally achieved using a path table. The table structure creation statement is as follows:

[0043]

[0044] Table structure and storage content examples are shown in Table 1:

[0045] Table 1: Example of question bank table structure and storage content

[0046] Primary Key User UID title Parent node primary key path 1 1 Root Node 0 1 2 1 Child Node 1 1 1-2 3 1 Child Node 2 2 1-2-3 4 1 Leaf Node 3 1-2-3-4

[0047] The present application provides a method and system for improving the speed of generating a personal question bank. On the basis of not changing the existing system table structure, a data preprocessing mechanism and a parallel processing method are adopted, which can greatly improve the speed of generating a personal question bank and improve the user experience.

[0048] Example 1

[0049] like Figure 3 As shown, the method for improving the speed of generating a personal question bank provided by the present application includes:

[0050] The remaining available number of preprocessed data is monitored at set intervals (eg, 5 minutes). When the remaining available number of preprocessed data is less than the set number (eg, 10 sets), the preprocessing process is executed.

[0051] The preprocessing process is: copy the "Official Question Bank-Chapter Tree" into the preprocessed data, and leave the "User UID" field blank or set it to specific characters for later association with the user.

[0052] When a user selects a chapter tree from the official question bank and copies it to a personal question bank, the "user UID" in the data of the chapter tree in the preprocessed data is directly modified to the user's UID, and the copying process is completed. After the copying is completed, the number of remaining available sets of the preprocessed data of the chapter tree is reduced by 1.

[0053] An example of preprocessed data is shown in Table 2:

[0054] Table 2: Example of preprocessed data

[0055] Primary Key User UID title Parent node primary key path 1 0 Root Node 0 1 2 0 Child Node 1 1 1-2 3 0 Child Node 2 2 1-2-3 4 0 Leaf Node 3 1-2-3-4

[0056] An example of data after the preprocessed data is associated with a user is shown in Table 3 (user UID=6):

[0057] Table 3: Example of preprocessed data after being associated with users

[0058] Primary Key User UID title Parent node primary key path 1 6 Root Node 0 1 2 6 Child Node 1 1 1-2 3 6 Child Node 2 2 1-2-3 4 6 Leaf Node 3 1-2-3-4

[0059] Compare the execution time of the technical solution of this application with that of the prior art solution:

[0060] Existing technical solution: When a user triggers a personal question bank and initiates data replication, the system needs to set a long loading waiting time for interaction. According to the execution time of a single sequence, the estimated waiting time is about 100 seconds.

[0061] 200*0.5=100(seconds)

[0062] The technical solution of this application: After the data pre-processing is completed, the user triggers the personal question bank, and only the user UID is associated with the data. The processing time is estimated to be within 1 second.

[0063] It can be seen that the performance of data processing by the technical solution of the present application is greatly improved, and the system user experience is good.

[0064] In order to further improve the speed of generating personal question banks, this application uses multi-process technology to perform the preprocessing process, that is, the chapter tree data processing under each grade and semester is processed using a separate process, such as Figure 4 shown.

[0065] There are many ways to implement the multi-process concurrent processing mechanism, such as:

[0066] 1. The multiprocessing module in Python implements parallel computing of multiple processors by creating and managing processes;

[0067] 2. The fork function call in Unix-like systems to create a new child process;

[0068] 3. Use the pcntl_fork function in PHP to create a child process.

[0069] Since tree-structured data storage has upstream and downstream dependencies, tree processing takes a long time. If a multi-process concurrent processing mechanism is adopted by grade and semester, a single chapter tree occupies one process to process, which can reduce the processing time of the entire academic semester.

[0070] According to the calculation of 200 official chapter trees involved in elementary school mathematics, the processing time of each tree is calculated as 500ms, and the execution time of parallel and serial is compared:

[0071] Time required for serial execution (sequential execution): 200*0.5=100 (seconds)

[0072] Time required for parallel execution (concurrent execution): approximately 0.5 (seconds)

[0073] It can be seen that using multiple processes to perform data replication in parallel can greatly improve the efficiency of single processing.

[0074] Example 2

[0075] The present application also provides a system for improving the speed of generating a personal question bank, which is implemented based on the above method and includes:

[0076] The monitoring module is used to monitor the remaining available sets of pre-processed data at set intervals (e.g., 5 minutes).

[0077] The preprocessing module is used to copy the official question bank to the preprocessing data when the remaining available number of preprocessing data is less than the set number (for example, 10 sets), and the "User UID" field is left blank or set to specific characters for later association with the user.

[0078] The parallel processing module is used to execute the preprocessing module using multi-process technology.

[0079] The present application may also provide a computer device, comprising: at least one processor, a memory, at least one network interface and a user interface. The various components in the device are coupled together through a bus system. It is understood that the bus system is used to achieve connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus and a status signal bus.

[0080] The user interface may include a display, a keyboard or a pointing device, such as a mouse, a trackball, a touch pad or a touch screen.

[0081] It is understood that the memory in the embodiments disclosed in the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0082] In some embodiments, the memory stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system and applications.

[0083] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present disclosure can be included in the application.

[0084] In the above embodiment, the processor may also call a program or instruction stored in the memory, specifically, a program or instruction stored in an application program, and is used to:

[0085] Execute the steps of the above method.

[0086] The above method can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The above-disclosed methods, steps and logic block diagrams can be implemented or executed. The general processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the above-disclosed method can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0087] It is understood that the embodiments described in the present application can be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0088] For software implementation, the technology of the present application can be implemented by executing the functional modules (such as procedures, functions, etc.) of the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0089] The present application may also provide a non-volatile storage medium for storing a computer program. When the computer program is executed by a processor, each step in the above method embodiment can be implemented.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that any modification or equivalent replacement of the technical solution of the present application does not depart from the spirit and scope of the technical solution of the present application and should be included in the scope of the claims of the present application.

Claims

1. A method for improving the speed of generating a personal question bank, which is suitable for generating a personal question bank with a tree structure, and the method includes: The remaining available number of pre-processed data is monitored at set intervals, and when the remaining available number of pre-processed data is less than the set number, the pre-processing process is executed; The preprocessing process is: copying the chapter tree of the official question bank into the preprocessing data, and leaving the information item identifying the user identity blank or setting it to a specific character.

2. The method for improving the speed of generating a personal question bank according to claim 1, characterized in that: The method further comprises: When a user selects a chapter tree from the official question bank and copies it to a personal question bank, the information item identifying the user's identity in the data of the chapter tree in the preprocessed data is directly replaced with the user's identity information.

3. The method for improving the speed of generating a personal question bank according to claim 2, characterized in that: The method further comprises: After completing the replacement of the information item identifying the user identity of the chapter tree in the pre-processed data, the number of remaining available sets of the pre-processed data of the chapter tree is reduced by 1.

4. The method for improving the speed of generating a personal question bank according to any one of claims 1 to 3, characterized in that: The method further comprises: Multi-process technology is used to perform the preprocessing process.

5. The method for improving the speed of generating a personal question bank according to claim 4, characterized in that: When the multi-process technology is used to perform the pre-processing process, the data of each chapter tree is copied using an independent process.

6. The method for improving the speed of generating a personal question bank according to claim 1, characterized in that: The setting time is 5 minutes.

7. The method for improving the speed of generating a personal question bank according to claim 1, characterized in that: The set number is 10.

8. A system for improving the speed of generating a personal question bank, implemented based on the method described in any one of claims 1 to 7, characterized in that: The system comprises: A monitoring module, used to monitor the remaining available number of pre-processed data at set intervals; and The preprocessing module is used to copy the official question bank into the preprocessing data when the remaining available number of preprocessing data is less than the set number of sets, and leave the information item identifying the user identity blank or set it to a specific character.

9. The system for improving the speed of generating a personal question bank according to claim 8, characterized in that: The system further comprises: The parallel processing module is used to execute the preprocessing module using multi-process technology.

Images