A teaching content defect positioning method and system and a storage medium

Abstract

The application discloses a teaching content defect positioning method and system and a storage medium. The method obtains a learning state record with a learner identifier, a teaching content identifier, a knowledge point identifier and a time window as a composite primary key; determines a prompt dependency category according to a minimum sufficient prompt level change amplitude, a same-structure different-content retest task retention rate under a zero prompt condition and error content sequence regularity test results, and counts a category frequency proportion of the same teaching content and the same knowledge point. When the proportion reaches a threshold value, a target regularity term first appearance step sequence and a regularity finding task completion step sequence are extracted from an interaction step sequence number table, a term advance step number is calculated, a to-be-confirmed defect record is output and added to a revision queue; subsequently, a variant content of the term is generated, new learners are shunted, and defects or rollback are confirmed according to the category frequency proportion difference of the two versions. The application can position the defect step sequence in the teaching content, and verify the positioning result.

Description

Technical Field

[0001] This invention relates to the field of data processing technology in computer-aided instruction, specifically to a method, system, and storage medium for automatically locating, verifying, and rolling back the internal defects of teaching content by utilizing the interaction status records of learner groups and the metadata of the interaction steps of teaching content. Background Technology

[0002] Existing methods for evaluating the quality of teaching resources typically take the resource itself and its historical response statistics as input, and output the overall difficulty, discrimination, knowledge point coverage, or quality labels of the resource. This type of output generally refers to the entire resource and cannot directly pinpoint the defects in any specific interactive step within the teaching content.

[0003] When the overall quality of resources is deemed poor, content designers still need to review each instance presentation, question, terminology presentation, system feedback, and pattern discovery task to manually determine the location of defects. As the resource library grows, the amount of manual review increases with the amount of content, and the review conclusions rely heavily on review experience.

[0004] While knowledge state estimation systems can output mastery probabilities or state labels based on learners' responses, this output is typically only used for task delivery to the learner. The state records generated by the learner group have not yet been structurally mapped back to the specific interaction sequences within the teaching content.

[0005] On the other hand, existing status tags are mostly bound to learner identifiers or learner identifiers and knowledge point identifiers. The statuses generated by the same learner for the same knowledge point on different teaching contents are easily merged, making it difficult to aggregate them according to teaching content identifiers to form content-side signals.

[0006] Learners may also overgeneralize during the pattern formation process. For example, after summarizing a general rule, learners may apply that rule to exceptions. While this type of error manifests as an increased error rate, it does not necessarily indicate a flaw in the teaching content. Failure to distinguish this type of pattern error can lead to systematic bias in content-based statistics.

[0007] Furthermore, the content defect labels output by a single statistical analysis may be affected by sample composition, learner background differences, or random fluctuations. Existing methods typically lack a closed loop for generating variant content, diverting new learners, comparing two versions, and confirming or rolling back defect labels. Summary of the Invention

[0008] While retaining the teaching content identifiers, extract content-side abnormal signals for specific teaching content and specific knowledge points from the learner group's status records; By combining the content-side abnormal signals with the teaching content interaction step sequence metadata, the specific step sequence number and defect type of the defect are automatically output. Eliminate regularity errors caused by overgeneralization of target patterns and reduce false positives in content defect statistics; Candidate defects are automatically verified through variant content generation, stable traffic distribution, comparative statistics, and rollback mechanisms. Reduce repetitive practice task query requests triggered by defect content during candidate defect verification. Technical solution

[0009] To address the aforementioned technical problems, this invention provides a computer-executed method for locating defects in teaching content.

[0010] Step A1 involves acquiring learning status records for multiple learners on the same teaching content and knowledge point. Each learning status record is bound to a composite primary key consisting of learner identifier, teaching content identifier, knowledge point identifier, and time window. When the minimum sufficient cue level changes little within the observation period, the retention rate of retesting tasks with different content of the same structure under zero-cue conditions is low, and the error content sequence regularity test does not output the first marker, the corresponding record is identified as the cue dependency category.

[0011] Step A2: Using teaching content identifiers and knowledge point identifiers as constraints, aggregate learner identifiers and calculate the frequency percentage of prompt dependency categories.

[0012] Step A3: When the frequency ratio reaches the threshold, read the interactive step sequence number table of the teaching content and extract the first occurrence step sequence number a of the target pattern term name and the completion step sequence number b of the pattern discovery task.

[0013] Step A4: Calculate the terminology advance step count d = ba. When d is greater than zero, generate a defect record with a confirmation status of "pending confirmation" and place the corresponding teaching content into the content revision queue.

[0014] Step A5 involves generating variant content where the term's first appearance is moved to a position after the pattern discovery task is completed. New learners are then stably distributed between the original teaching content and the variant content, and the frequency percentage of the cue dependency category is calculated for each. When the decrease in the variant content relative to the original teaching content reaches a threshold, the defect is confirmed and the variant content is activated; otherwise, the defect status is set to unconfirmed and a rollback is performed.

[0015] The present invention also provides a teaching content defect location system, including a status acquisition module, a group statistics module, a metadata reading module, a defect output module, a revision verification module, a storage module, a teaching content metadata storage unit, a content revision queue, and a distribution unit.

[0016] The present invention also provides a computer-readable storage medium on which a computer program stored is executed by a processor to implement the above-described method. Beneficial effects

[0017] This invention outputs the specific defect sequence, defect type, and terminology advance steps within the teaching content, rather than simply outputting the quality label of the entire resource.

[0018] Learning status records are stored using a composite primary key that includes teaching content identifiers. This allows for aggregation of content-side signals based on teaching content identifiers and knowledge point identifiers, while retaining the data foundation for aggregating learner-side status distributions based on learner identifiers.

[0019] By examining the regularity of error content sequences, overgeneralization errors can be eliminated, thus avoiding misclassifying systematic errors that occur during the formation of patterns as content defects.

[0020] Candidate defects are first written to the defect record in a pending confirmation state, and then verified through variant content and new learners. If the verification result does not support the candidate defect, the process is automatically rolled back, thereby reducing false positives.

[0021] During the verification process, repeated practice query requests triggered by specific teaching content identifiers and knowledge point identifiers are suppressed, reducing invalid question bank retrieval and transmission.

[0022] For multiple sets of teaching materials that point to the same knowledge point, the number of steps the term is advanced, the frequency ratio, and the correlation coefficient between the two can be calculated to achieve cross-content comparison. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the module structure of the system of the present invention.

[0024] Figure 2 This is a flowchart illustrating the overall process of the method of the present invention.

[0025] Figure 3 This is a schematic diagram of the metadata structure and interactive steps of the teaching content.

[0026] Figure 4 A flowchart for calculating the number of steps in advance for terminology and outputting defect records.

[0027] Figure 5 Flowchart for the verification, confirmation, and rollback of variant content.

[0028] Figure 6 This is a schematic diagram of how learning state records are stored with a composite primary key and aggregated in two directions.

[0029] Figure 7 This diagram illustrates the distribution, ranking, and correlation of terms in multiple teaching materials that refer to the same knowledge point, based on their advance steps, frequency percentages, and correlations.

[0030] Figure labeling: 201—Status acquisition module; 202—Group statistics module; 203—Metadata reading module; 204—Defect output module; 205—Revision verification module; 206—Storage module; 207—Teaching content metadata storage unit; 208—Content revision queue; 209—Distribution unit. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments. These embodiments are used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0032] Teaching content consists of executable content units composed of multiple interactive events presented in sequence, each with a unique step number. Interactive events include instance presentation, questioning, learner responses, system feedback, terminology presentation, and pattern discovery tasks.

[0033] Target pattern: The input-output mapping relationship corresponding to knowledge points, which can be described by a finite set of rules.

[0034] Terminology: The standard term for the target law within the corresponding discipline system.

[0035] Pattern discovery task: This task requires learners to summarize the target pattern through interactive events involving observation and verification of examples, without being informed of the terminology used. The completion criteria for the pattern discovery task may include learners outputting a statement equivalent to the target pattern, or answering correctly a preset number of times on the verification task.

[0036] Minimum sufficient cue level: The cue level at which the learner first independently completes a task on a homogeneous adjacent task during the process of the cue controller applying cues step by step from the lowest level; when the task can be completed independently without cues, it is recorded as level zero.

[0037] Retention rate: The ratio of the number of times a learner passes a retest task with the same structure but different content as the original task after a preset interval period, without any level prompts.

[0038] First label: The label output when the erroneous items in the erroneous content sequence can be uniformly explained by the overgeneralization rule of the target pattern.

[0039] Hint Dependency Category: The learning state category determined when the minimum sufficient cue level changes by less than a preset change threshold, the retention rate is less than a preset retention rate threshold, and no first tag is output.

[0040] Term advance step count: The difference between the step number b when the pattern discovery task is completed and the step number a when the term name first appears, i.e., d = ba. A d greater than zero indicates that the term name appears before the pattern discovery task is completed.

[0041] Confirmation Status: The status field of the defect record, including pending confirmation, confirmed, and unconfirmed.

[0042] See Figure 1 The status acquisition module 201 reads the learning status records stored by composite primary key from the storage module 206, or receives the records output in real time by the learning status generation program; the group statistics module 202 performs content-side aggregation on the records; the metadata reading module 203 reads the interaction step sequence data from the teaching content metadata storage unit 207; the defect output module 204 generates defect records and writes them to the content revision queue 208; the revision verification module 205 generates variant content and calls the diversion unit 209 for comparison verification; the storage module 206 saves the learning status records, defect records and verification results. Figure 1 The arrow pointing from the storage module 206 to the status acquisition module 201 indicates the reading direction of the status record.

[0043] The learning status record includes at least the fields learner_id, content_id, kp_id, time_window, state_class, and marker. Among them, learner_id, content_id, kp_id, and time_window together form a composite primary key.

[0044] In one implementation, the cue controller applies cues in ascending order of intensity, prohibiting the application of higher-intensity cues before the current level has been applied. Each level of cue does not directly provide a terminological name for the target pattern. The system records the lowest sufficient cue level the learner achieves when independently completing a homogeneous adjacent task for the first time.

[0045] After an interval of no less than 24 hours from the time the original task was completed, the system schedules retest tasks with the same structure but different content and calculates the retention rate under zero-prompt conditions.

[0046] In this embodiment, the observation duration ranges from 14 to 60 days, preferably 28 days; the minimum sufficient warning level change threshold is 1 warning level; and the retention rate threshold ranges from 0.40 to 0.60, preferably 0.50.

[0047] The system derives the error generation rule Re from each error item in the error content sequence and calculates the proportion C of the number of error items that can be explained by the same Re to the total number of error items.

[0048] If Re's output on the positive example set of the target rule Rt is consistent with Rt, and only inconsistent with Rt on the exception set of Rt, and C is not less than the third threshold, then the first label is output. Records with the first label are not determined as cue dependency categories. The third threshold ranges from 0.60 to 0.90, preferably 0.70.

[0049] If the output of the error generation rule on the set of positive examples of the target rule is inconsistent with the target rule, then a second label or no first label result can be output. This result can be used as input for prompting dependency category determination and other diagnostic processes.

[0050] The group statistics module 202 aggregates learner_id based on content_id and kp_id as constraints. Let the total number of effective learners be N, and the number of learners identified as belonging to the cue dependency category be Nc. Then the frequency percentage Qc = Nc / N.

[0051] The first threshold ranges from 0.25 to 0.40, preferably 0.30. The metadata reading step only proceeds when the total number of effective learners reaches the preset minimum sample size and Qc is not less than the first threshold.

[0052] See Figure 3 The metadata for each teaching content M includes: content_id, representing the teaching content identifier; kp_id, representing the knowledge point identifier; step_table, representing the interaction step sequence number table that records the interaction event type and sequence number; term_first_step, representing the step sequence number a of the first occurrence of the term name; discovery_done_step, representing the step sequence number b of the completion of the pattern discovery task; positive_count, representing the total number of positive instances; and exception_first_step, representing the step sequence number of the first occurrence of the exception set instance.

[0053] term_first_step, discovery_done_step, positive_count, and exception_first_step can be automatically extracted by the program based on step_table during content compilation or publication and written into the teaching content metadata storage unit 207.

[0054] See Figure 4 The metadata reading module 203 reads a and b, and the defect output module 204 calculates d=ba. When d is greater than zero, a defect record of the defect type "premature term disclosure" is generated. The step number where the defect is located is a, the premature term disclosure step number is d, and the initial confirmation status is set to pending confirmation.

[0055] The defect record includes at least the content_id, kp_id, defect type, step number where the defect occurred, the number of steps the terminology was advanced, and the confirmation status. The defect output module 204 writes the corresponding teaching content identifier into the content revision queue 208.

[0056] Teaching content M1 corresponds to the knowledge point "subject-verb agreement recognition". Steps 1 and 2 are examples, step 3 is terminology, steps 4 to 10 include examples, questions, learner answers and system feedback, step 11 is the completion of the pattern discovery task, and steps 12 and above are practice tasks.

[0057] Therefore, we get a=3, b=11, d=8.

[0058] Statistical analysis was conducted on 320 effective learners using M1, of whom 156 met both the minimum sufficient cue level change range and retention rate criteria. Among these 156 learners, 21 had their erroneous content sequences outputting the first label, and therefore were not identified as cue-dependent categories. The final Nc=135, Qc=135 / 320=0.421875, rounded to two decimal places as 0.42.

[0059] Let the first threshold be 0.30. Since Qc is not less than the first threshold and d is greater than zero, the system outputs a record of unconfirmed defects in step 3 of teaching content M1, indicating premature disclosure of terminology, and places M1 into content revision queue 208.

[0060] See Figure 5 The system copies the original teaching content M and generates variant content M′. Except for moving the terminology presentation event to after the completion sequence of the pattern discovery task, M′ keeps the content, relative order, interface parameters, and task difficulty of other interactive events unchanged.

[0061] The splitting unit 209 calculates a hash value based on the newly added learner identifier and stably maps learners to M and M′ according to a preset ratio. The same learner remains in the same version during verification to avoid cross-version contamination.

[0062] Let p be the frequency percentage of the suggestion dependency category on M, and p′ be the frequency percentage on M′. When p is greater than zero, the decrease is G = (pp′) / p. The second threshold ranges from 0.20 to 0.50, preferably 0.30. If G is not less than the second threshold, the defect record confirmation status is set to confirmed, and M′ is set to effective content; otherwise, the confirmation status is set to unconfirmed, M is removed from the content revision queue, and M′ is no longer considered as a candidate replacement content.

[0063] When p equals zero, or when the effective sample size of any shunting group is less than the preset minimum sample size, G is not calculated, the confirmation status remains as pending confirmation, and sample collection continues.

[0064] Move the terminology presentation event from step 3 to step 12 of M1 to generate variant content M1′. Add 300 learners and split them in a 1:1 ratio: 150 in group M1 and 150 in group M1′.

[0065] The frequency percentage of the dependency category in group M1 is 0.40, and the frequency percentage of group M1′ is 0.11, with a decrease of G = (0.40 - 0.11) / 0.40 = 0.725. Assuming the second threshold is 0.30, since 0.725 is not less than 0.30, the system sets the defect record confirmation status to "confirmed" and sets M1′ to "effective".

[0066] If the decrease in value does not reach the second threshold, the candidate defect is not confirmed, and the system performs a rollback. Therefore, the statistical phase outputs the candidate defects, and the triage and verification phase outputs the final confirmed results.

[0067] Insufficient Instances Defect: When positive_count is less than the fourth threshold, an insufficient instance flag is output, and discovery_done_step is recorded as the defect-related step number. In this embodiment, the fourth threshold ranges from 5 to 12, preferably 8.

[0068] Premature introduction of defects: When exception_first_step is less than discovery_done_step, output a premature introduction of defects flag and record exception_first_step as the step number where the defect is located.

[0069] All the above markers are written into a unified defect record structure. Different defect types can generate corresponding variant content, which can then be verified according to step A5.

[0070] See Figure 6 The learning status record uses learner_id, content_id, kp_id, and time_window as a composite primary key, and stores fields such as state_class and marker.

[0071] Constraining content_id and kp_id, and aggregating learner_id, yields the content-side frequency percentage of a certain teaching content on a certain knowledge point; constraining learner_id and kp_id, and aggregating content_id, yields the state distribution of a learner across multiple teaching contents on the same knowledge point.

[0072] Figure 1 The storage module 206 provides the above records to the status acquisition module 201; the group statistics module 202 executes step A2 in the content-side direction.

[0073] See Figure 7For multiple teaching materials Ma to Mf that point to the same knowledge point K7, calculate the term advance steps and the frequency percentage of prompt dependency categories respectively.

[0074] For example, the advance steps for terms Ma, Mb, Mc, Md, Me, and Mf are 14, 9, 8, 2, 0, and -3 respectively, corresponding to frequency percentages of 0.47, 0.38, 0.42, 0.19, 0.14, and 0.09. The system outputs the sorted results by the advance steps for terms and calculates the correlation coefficient between the advance steps for terms and the frequency percentage, writing this correlation coefficient into the defect record of knowledge point K7.

[0075] From the moment the teaching content is placed into the content revision queue until the confirmation status changes to confirmed or unconfirmed, the task scheduler sets a suppression flag for repetitive practice task query requests triggered by the content_id and kp_id.

[0076] The suppression does not affect input tasks containing the target pattern, tasks on other teaching content, or variant content tasks used for verification. By limiting the trigger key, duplicate question bank retrieval and data transmission for the same defective content can be reduced.

[0077] Learning status categories can be generated locally on the learner's terminal; only the `state_class`, marker, and composite primary key are uploaded to the server. Original responses can be stored on the terminal. The server performs group statistics, metadata retrieval, defect output, and revision verification.

[0078] The output object of this invention is teaching content, and the output result is the location, type and verification status of candidate or confirmed defects within the teaching content. No evaluative conclusions are output for individual learners.

[0079] The system does not collect biometric data such as camera images, facial expressions, eye movements, or heart rate. When processing interactive data from minors, it only processes the minimum amount of data required to implement the above methods.

[0080] This invention is not intended for the diagnosis, prevention, monitoring, treatment or relief of diseases, nor for the examination or replacement of physiological processes.

[0081] The above embodiments are used to illustrate the technical solution of the present invention. Equivalent substitutions, combinations, or improvements made by those skilled in the art without departing from the spirit and principles of the present invention may all fall within the protection scope of the present invention.

Claims

1. A method for locating deficiencies in teaching content, executed by a computer, characterized in that, include: A1. Obtain learning status records of multiple learners on the same knowledge point of the same teaching content; each learning status record is bound to a composite primary key consisting of learner identifier, teaching content identifier, knowledge point identifier and time window; When the change in the minimum sufficient cue level within the preset observation period is less than the preset change threshold, the retention rate on the same but different retest task without applying any cue level is less than the preset retention rate threshold, and the regularity test of the error content sequence does not output the first label to indicate overgeneralization error, the corresponding learning state record is determined as the cue dependency category. A2. Using the teaching content identifier and the knowledge point identifier as constraints, aggregate the learner identifiers and count the frequency percentage of learners identified as belonging to the prompting dependency category among the multiple learners. A3. When the frequency ratio is not less than the first threshold, read the interaction step sequence number table of the teaching content, and extract the step sequence number a of the first appearance of the term name of the target rule corresponding to the knowledge point, and the step sequence number b of the completion of the rule discovery task from the interaction step sequence number table; the rule discovery task is a task that requires learners to summarize the target rule by observing and verifying examples without being told the term name. A4. Calculate the advance step count for terms d = ba; when d is greater than zero, generate a defect record containing teaching content identifier, knowledge point identifier, defect type, defect sequence number a, advance step count for terms d, and confirmation status, set the confirmation status to pending confirmation, and place the teaching content into the content revision queue. A5. Generate variant content of the teaching content, wherein the first occurrence order of the term name in the variant content is adjusted to after the completion order of the pattern discovery task; new learners are diverted to the teaching content and the variant content, and the frequency ratio of the prompt dependency category of each is calculated; when the frequency ratio of the variant content decreases by no less than a second threshold relative to the teaching content, the confirmation status is set to confirmed and the variant content is set to effective content; otherwise, the confirmation status is set to unconfirmed and the teaching content is removed from the content revision queue.

2. The method according to claim 1, characterized in that, The regularity test in step A1 includes: summarizing error generation rules from each error item in the error content sequence; calculating the proportion of error items that can be explained by the same error generation rule to the total number of error items, as a consistency index; when the output of the error generation rule on the positive example set of the target rule is consistent with the target rule, and only the output on the exception set of the target rule is inconsistent with the target rule, and the consistency index is not less than a third threshold, the first label is output; when the first label is output, the corresponding learning state record is not determined as the prompt dependency category.

3. The method according to claim 1, characterized in that, The metadata of the teaching content also includes at least one of the following: the total number of positive instances of the target rule, the step sequence number of the first occurrence of the exception set instance of the target rule, and the interaction step sequence number table that records the type and sequence number of each interaction event.

4. The method according to claim 3, characterized in that, It also includes outputting at least one of the following defect types: when the total number of positive instances is less than the fourth threshold, outputting an insufficient instance flag and recording the completion step number of the pattern discovery task as the defect-associated step number; when the step number of the first occurrence of the exception set instance is less than the completion step number of the pattern discovery task, outputting an premature exception flag and recording the first occurrence step number of the exception set instance as the defect-related step number; and writing the insufficient instance flag or the premature exception flag into the defect record of the teaching content.

5. The method according to claim 1, characterized in that, In step A5: Calculate the hash value based on the learner identifier of the new learner, and map the new learner stably to the teaching content and the variant content according to the hash value at a preset ratio; let p be the frequency ratio of the cue dependency category of the teaching content and p′ be the frequency ratio of the cue dependency category of the variant content; when p is greater than zero, the decrease is (pp′) / p; when p is equal to zero or the effective sample size of any split group is less than the preset sample size, keep the confirmation status as pending confirmation until the calculation conditions are met.

6. The method according to claim 1, characterized in that, The learning status record also includes a status category field and a tag field; in step A2, by constraining the teaching content identifier and knowledge point identifier and aggregating the learner identifier, the frequency ratio of the content side is obtained; by constraining the learner identifier and knowledge point identifier and aggregating the teaching content identifier, the status distribution of the learner across multiple teaching contents on the same knowledge point is obtained.

7. The method according to claim 6, characterized in that, Also includes: Obtain multiple sets of teaching materials that point to the same knowledge point; Perform steps A1 to A4 respectively to obtain the number of advance steps for terminology and the frequency percentage of prompt dependency categories for each set of teaching content; Sort the multiple teaching materials according to the aforementioned terms and advance steps, and output the sorting results; Calculate the correlation coefficient between the advance step count of the term and the frequency ratio, and write the correlation coefficient into the defect record of the knowledge point.

8. The method according to claim 1, characterized in that, From the moment the teaching content is placed into the content revision queue until the confirmation status is set to confirmed or unconfirmed, the repetitive practice task query requests triggered by the teaching content identifier and the knowledge point identifier are suppressed, and input tasks containing the target pattern are maintained.

9. A teaching content defect location system, characterized in that, include: The status acquisition module is used to acquire the learning status records of multiple learners on the same knowledge point of the same teaching content, and to determine the prompt dependence category based on the minimum sufficient prompt level change range, the retention rate of retest tasks with different content of the same structure under zero prompt conditions, and the results of the error content sequence regularity test. The group statistics module is used to count the frequency percentage of the prompt dependency category; The metadata reading module is used to read the interaction step sequence number table from the teaching content metadata storage unit when the frequency ratio reaches the threshold, and extract the first occurrence step sequence number of the term name and the completion step sequence number of the pattern discovery task; the defect output module is used to calculate the number of steps ahead of the term, generate defect records with the confirmation status as pending confirmation, and put the teaching content into the content revision queue. The revision verification module is used to generate variant content, and the new learners are diverted to the original teaching content and variant content through the diversion unit. Based on the difference in the frequency ratio of the prompt dependency categories of the two, defects are identified or rollbacks are performed. The storage module is used to store the learning status records with a composite primary key consisting of learner identifier, teaching content identifier, knowledge point identifier and time window.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images