Learning behavior identification method and system based on edge calculation

Abstract

The invention discloses a learning behavior identification method and system based on edge computing, and the method comprises the steps: marking a pause event in a teaching video, and generating a pause event broken line graph; performing broken line smoothing on the pause event broken line graph to generate a pause event curve graph; in the pause event curve graph, determining M knowledge point fragments of the teaching video; obtaining a reference behavior vector sequence of J reference objects according to the M knowledge point fragments of the teaching video; acquiring a real-time behavior vector sequence of the to-be-identified object, and identifying the most similar object of the to-be-identified object from the J reference objects; determining learning behavior evolution distribution of the to-be-identified object according to the most similar object and the reference behavior vector sequence thereof, and matching a pre-trained learning intervention strategy based on the learning behavior evolution distribution; according to the method, the characteristics of the pause event associated with the knowledge point fragment are extracted, so that the stagnation behavior of the reference object in the specific teaching content can be captured, and the knowledge point fragment in the learning process can be identified.

Description

Technical Field

[0001] The present invention relates to the field of behavior recognition, and specifically to a learning behavior recognition method based on edge computing. Background Art

[0002] With the popularization of online intelligent teaching systems, the analysis of learning process behaviors based on teaching videos has become an important means to improve personalized teaching services. In the prior art, learning behavior recognition usually relies on modeling explicit behaviors such as clicks, plays, and pauses during the learning process to infer the learning state and cognitive load level.

[0003] Traditional methods generally use a centralized server for data collection and processing, which has problems of insufficient real-time performance and lagging feedback. For this reason, some technical solutions propose to apply edge computing to the local collection and preliminary processing of learning behavior data, aiming to improve the timeliness and response speed of learning behavior analysis. However, the prior art still has the following deficiencies: First, existing methods mostly use overall statistical features (such as play completion rate, total pause times) as the basis for learning state evaluation, lacking a dynamic knowledge point division mechanism based on the fine-grained changes of pause behaviors, and unable to automatically identify potential cognitive load concentration positions during the learning process; Second, although it can quickly process learning behavior data on the edge side, it lacks a similarity matching method based on temporal behavior features and is unable to dynamically infer the learning evolution trend of the object to be recognized according to the learning trajectory of historical reference objects.

[0004] Therefore, there is an urgent need for a learning behavior recognition method and system that can collect fine-grained pause behaviors in real time on the learning terminal side, combine with the teaching content time axis, automatically divide knowledge point segments, and identify the most similar reference objects, so as to support personalized learning intervention. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides a learning behavior recognition method based on edge computing, which solves the technical problems raised in the background art by automatically dividing knowledge point segments and identifying the most similar reference objects.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: A learning behavior recognition method based on edge computing, the behavior recognition method includes: S1. Obtain a teaching video and its teaching time axis, and mark the pause events that occur in the teaching video on the teaching time axis; S2. Generate a line graph of pause events of the teaching video according to the pause events that occur in the teaching video; S3. Smooth the line graph of pause events to generate a curve graph of pause events; S4. Determine M knowledge point segments of the teaching video in the pause event line graph; S5. Obtain the reference behavior vector sequences of J reference objects according to the M knowledge point segments of the teaching video; S6. Obtain the real-time behavior vector sequence of the object to be recognized, and identify the most similar object of the object to be recognized from the J reference objects; S7. Determine the learning behavior evolution distribution of the object to be recognized according to the most similar object and its reference behavior vector sequence, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

[0007] In some of the embodiments, generating a pause event line graph of the teaching video according to the pause events occurring in the teaching video S2-1. Divide the teaching timeline into N time sub-axes of the same length, and assign an original time serial number based on the time sequence to each time sub-axis; S2-2. Obtain several pause events of the teaching video within each time sub-axis; Among them, each pause event is associated with its original time serial number, student identifier, timestamp of this pause, and pause duration to form a set of pause events within the time sub-axis; S2-3. Construct an initial coordinate graph with the teaching timeline as the X-axis and the number of pause events as the Y-axis; S2-4. Mark the number of pause events of the teaching video within each time sub-axis in the initial coordinate graph to generate N pause points; S2-5. Connect any adjacent pause points to generate the pause event line graph of the teaching video.

[0008] In some of the embodiments, performing line smoothing on the pause event line graph to generate a pause event curve graph, including: S3-1. Mark three arbitrarily adjacent time sub-axes; S3-2. Obtain the original time serial numbers of three arbitrarily adjacent time sub-axes, and define the time sub-axis corresponding to the smallest original time serial number as the anchor axis; S3-3. Obtain the total number of pause events of the three arbitrarily adjacent time sub-axes; S3-4. Calculate the average number of pause events according to the total number of pause events of the three arbitrarily adjacent time sub-axes; S3-5. Define the average number of pause events as the smoothed number of pause events of the anchor axis; S3-6. Repeat S3-1 to S3-5 until the smoothed number of pause events of each time sub-axis is obtained; S3-7. Mark the smoothed number of pause events of each time sub-axis to generate N smoothed pause points; S3-8. Connect any adjacent smooth pause points to generate a pause event curve graph of the teaching video.

[0009] In some of these embodiments, in the pause event curve graph, determine M knowledge point segments of the teaching video, including: S4-1. Mark the knowledge point peaks in the K smooth pause point sets in the pause event curve graph; S4-2. Extend dividing lines along the K knowledge point peaks to the teaching time axis to divide the teaching video into M knowledge point segments; where M = K + 1.

[0010] In some of these embodiments, marking the knowledge point peaks in the K smooth pause point sets in the pause event curve graph includes: S4-1-1. In the pause event curve graph, select the number of smooth pauses corresponding to each time sub-axis; S4-1-2. Determine whether the number of smooth pauses corresponding to each time sub-axis is greater than the number of smooth pauses of its two adjacent time sub-axes before and after; S4-1-3. If it is greater, mark the smooth pause point as a candidate peak in the pause event curve graph; S4-1-4. Repeat S4-1-2 to S4-1-3 until all candidate peaks in the pause event curve graph are marked; S4-1-5. Calculate the time distance between adjacent candidate peaks among all candidate peaks; S4-1-6. If the time distance is less than the set minimum threshold, mark the larger peak among the adjacent candidate peaks as a knowledge point peak, otherwise, mark both adjacent candidate peaks as knowledge point peaks until K knowledge point peaks are obtained.

[0011] In some of these embodiments, according to the M knowledge point segments of the teaching video, obtain the reference behavior vector sequences of J reference objects, including: S5-1. Mark the covered pause events on several time sub-axes involved in the knowledge point segment; S5-2. Use the original time serial number of the associated time sub-axis in the covered pause event as the time sequence position reference, determine the start and end order of the knowledge point segment on the teaching time axis, and assign consecutive increasing advanced time serial numbers to each knowledge point segment based on this start and end order; Among them, the advanced time serial number is used to represent the time sequence position of the knowledge point segment in the teaching video; S5-3. Construct a reference behavior vector indexed by the student identifier from the covered pause events; Among them, the features of the reference behavior vector include: student identification, advanced time serial number, timestamp of each pause, pause duration, and total pause duration; S5-4. Obtain the reference behavior vectors of M knowledge point segments of the reference object in the teaching video. S5-5. Sort the reference behavior vectors of the M knowledge point segments according to the advanced time serial numbers corresponding to the M knowledge point segments to generate a personalized reference behavior vector sequence. Among them, the reference behavior vector sequence is used to represent the personalized learning trajectory formed by the reference object on the teaching timeline. S5-6. Select J reference objects and obtain the reference behavior vector sequences of the J reference objects.

[0012] In some embodiments, obtaining the real-time behavior vector sequence of the object to be recognized and identifying the most similar object of the object to be recognized from the J reference objects includes: S6-1. Obtain the real-time behavior vectors of the first Q knowledge point segments of the object to be recognized in the same teaching video and the first Q advanced time serial numbers. S6-2. Sort the real-time behavior vectors of the first Q knowledge point segments according to the first Q advanced time serial numbers to generate the real-time behavior vector sequence of the object to be recognized. S6-3. Determine the most similar object of the object to be recognized from the J reference objects according to the real-time behavior vector sequence of the object to be recognized.

[0013] In some embodiments, determining the most similar object of the object to be recognized from the J reference objects according to the real-time behavior vector sequence of the object to be recognized includes: S6-3-1. Extract the first Q reference behavior vectors from the reference behavior vector sequences of the J reference objects according to the advanced time serial numbers. S6-3-2. For each reference object, pair the real-time behavior vector with the same advanced time serial number with the reference behavior vector to form Q groups of time series vector pairs. S6-3-3. Calculate the similarity of the time series vector pairs in sequence according to the order of the advanced time serial numbers. If the similarity is less than the set threshold, terminate the matching of the current reference object and jump to the remaining reference objects; otherwise, calculate the similarity of the next group of time series vector pairs. S6-3-4. If the similarities of the current Q groups of time series vector pairs are all greater than the set threshold, mark this reference object as the most similar object.

[0014] In some embodiments, determining the learning behavior evolution distribution of the object to be recognized according to the most similar object and its reference behavior vector sequence, and matching the pre-trained learning intervention strategy based on the learning behavior evolution distribution includes: S7-1. Select the reference behavior vector sequence of the most similar object from the reference behavior vector sequences of J reference objects. S7-2. Extract the pause timestamps and pause durations of the most similar object at each knowledge point segment from the reference behavior vector sequence of the most similar object, and calculate the learning difficulty of each knowledge point segment until the learning difficulties of M knowledge point segments are obtained. S7-3. Construct the learning behavior distribution of the most similar object according to the pause timestamps, pause durations and learning difficulties of M knowledge point segments. S7-4. Define the learning behavior evolution distribution of the most similar object as the learning behavior evolution distribution of the object to be recognized, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

[0015] In the present invention, by collecting the pause events of the reference object in real time on the teaching timeline during the video playback, constructing a pause event curve graph, and automatically dividing the knowledge point segments in the teaching video based on the curve peak detection. Compared with the prior art that requires manual division or coarse-grained feature division, the present invention can accurately capture the positions where the cognitive load is concentrated based on the change of the pause density of the learner group on the teaching timeline, so as to automatically identify the knowledge point boundaries in the learning process, improving the fine-grainedness and accuracy of learning behavior analysis.

[0016] Furthermore, the present invention performs local collection and processing of pause events on the learning terminal side, improving the real-time response ability of learning behavior recognition. On this basis, by constructing the learning behavior vector sequence of the reference object and based on the continuous time series similarity matching mechanism, the reference object most similar to the object to be recognized is identified, realizing the dynamic inference of the learning behavior evolution trajectory. Combined with the pre-trained learning intervention strategy, personalized learning intervention can be timely pushed for the object to be recognized according to the learning difficulty distribution corresponding to each knowledge point segment.

[0017] In a second aspect, the present invention provides a learning behavior recognition system based on an edge computing architecture, including: A teaching video acquisition module, configured to acquire a teaching video and its teaching timeline, and mark the pause events occurring in the teaching video on the teaching timeline; A line graph generation module, configured to generate a pause event line graph of the teaching video according to the pause events occurring in the teaching video; A curve graph generation module, configured to smooth the line of the pause event line graph to generate a pause event curve graph; A knowledge point segment determination module, configured to determine M knowledge point segments of the teaching video in the pause event curve graph; A reference sequence acquisition module, configured to acquire the reference behavior vector sequences of J reference objects according to the M knowledge point segments of the teaching video. The most similar object recognition module is used to obtain the real-time behavior vector sequence of the object to be recognized and recognize the most similar object of the object to be recognized from J reference objects; The learning behavior intervention module is used to determine the learning behavior evolution distribution of the object to be recognized according to the most similar object and its reference behavior vector sequence, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

[0018] Compared with the prior art, the beneficial effects of a learning behavior recognition system based on an edge computing architecture of the present invention are the same as those of the above-mentioned learning behavior recognition method based on edge computing, so they will not be elaborated here. Description of the Drawings

[0019] Figure 1 It is a schematic flowchart of a learning behavior recognition method based on edge computing of the present invention; Figure 2 It is a schematic flowchart of the marking process of the knowledge point peak described in the present invention; Figure 3 It is a schematic flowchart of the marking process of the most similar object described in the present invention; Figure 4 It is a structural block diagram of a learning behavior recognition system based on edge computing of the present invention. Detailed Embodiments

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] Please refer to Figures 1 to 3 , the present invention provides a learning behavior recognition method based on edge computing, and the behavior recognition method includes the following steps: S1. Obtain the teaching video and its teaching timeline, and mark the pause events that occur in the teaching video on the teaching timeline; Among them, the pause event represents the operation of interrupting the playback of the reference object during the playback of the teaching video, and includes information such as the pause timestamp, the pause duration, the student identifier, and the serial number of the time sub-axis to which it belongs; moreover, the pause event can be recorded and uploaded in real time by an edge device (such as a teaching terminal or the client of a learning platform) to support behavior analysis based on an edge computing architecture.

[0022] S2. Generate a pause event line chart of the teaching video according to the pause events that occur in the teaching video; Exemplarily, step S2 specifically further includes: S2-1. Divide the teaching timeline into N time sub-axes of the same length, and assign an original time serial number based on the time sequence to each time sub-axis; S2-2. Obtain several pause events of the teaching video within each time sub-axis; Among them, each pause event is associated with its original time serial number, student identifier, timestamp of this pause, and pause duration to form a set of pause events within the time sub-axis; S2-3. Construct an initial coordinate graph with the teaching timeline as the X-axis and the number of pause events as the Y-axis; S2-4. Mark the number of pause events of the teaching video within each time sub-axis in the initial coordinate graph to generate N pause points; S2-5. Connect any adjacent pause points to generate a broken line graph of the pause events of the teaching video.

[0023] The initial coordinate graph uses the teaching timeline as the horizontal axis (X-axis) and the number of detected pause events within each time sub-axis as the vertical axis (Y-axis) to mark the change of the pause time of each time sub-axis. Connect the pause points corresponding to any two adjacent time sub-axes with a straight line in sequence to form a discontinuous broken line, thereby constructing the change trend of the pause time of the group object during the entire playback period of the teaching video. Therefore, the broken line graph of the pause events can reflect the density of the pause behavior of the reference object caused by the content of the teaching video in different time periods.

[0024] Further, the recognition method further includes: S3. Smooth the broken line of the broken line graph of the pause events to generate a curve graph of the pause events; The broken line graph is an original discontinuous distribution graph with the time sub-axis as the horizontal axis and the number of pause events as the vertical axis, which is used to characterize the temporal density change of the pause behavior during the group learning process; the curve graph is a continuous curve graph after smoothing on the basis of the broken line graph, which is used to eliminate the accidental fluctuations of the pause points and improve the stable recognition degree of the high-frequency pause positions.

[0025] Exemplarily, step S3 specifically further includes: S3-1. Mark three arbitrarily adjacent time sub-axes; S3-2. Obtain the original time serial numbers of the three arbitrarily adjacent time sub-axes, and define the time sub-axis corresponding to the smallest original time serial number as the anchor axis; S3-3. Obtain the total number of pause events of the three arbitrarily adjacent time sub-axes; S3-4. Calculate the average number of pause events according to the total number of pause events of the three arbitrarily adjacent time sub-axes; S3-5. Define the average number of pause events as the smooth pause count of the anchor axis; S3-6. Repeat S3-1 to S3-5 until the smooth pause counts of each time sub-axis are obtained; S3-7. Mark the smooth pause counts of each time sub-axis to generate N smooth pause points; S3-8. Connect any adjacent smooth pause points to generate a pause event curve graph of the teaching video.

[0026] In this embodiment, by averaging the total number of pause events of every three consecutive time sub-axes to generate the smooth pause count, a continuous pause event curve graph is constructed on the basis of the pause event line graph. Through local window smoothing processing, the local severe fluctuations caused by the occasional pause behaviors of individual students can be eliminated, and the interference of random noise on the overall time series distribution can be reduced.

[0027] Further, the recognition method further includes: S4. In the pause event curve graph, determine M knowledge point segments of the teaching video; Exemplarily, step S4 specifically further includes: S4-1. Mark the knowledge point peaks in the K smooth pause point sets in the pause event curve graph; S4-2. Extend dividing lines along the K knowledge point peaks to the teaching time axis to divide the teaching video into M knowledge point segments; where M = K + 1.

[0028] Among them, the knowledge point peak is characterized as: the time position where a number of reference objects centrally have pause behaviors during the learning process of the teaching video, and the teaching content at this time position may mean the existence of cognitive resistance. Based on this, this time position is used as the basis for dividing the boundaries of teaching knowledge points.

[0029] Among them, step S4-1 further includes: S4-1-1. In the pause event curve graph, select the smooth pause count corresponding to each time sub-axis; S4-1-2. Judge whether the smooth pause count corresponding to each time sub-axis is greater than the smooth pause counts of its two adjacent time sub-axes before and after; S4-1-3. If it is greater, mark the smooth pause point as a candidate peak in the pause event curve graph; S4-1-4. Repeat S4-1-2 to S4-1-3 until all candidate peaks in the pause event curve graph are marked; S4-1-5. In all candidate peaks, calculate the time distance between adjacent candidate peaks; S4-1-6. If the time distance is less than the set minimum threshold, mark the larger peak among adjacent candidate peaks as the knowledge point peak; otherwise, mark both adjacent candidate peaks as knowledge point peaks until K knowledge point peaks are obtained.

[0030] In this embodiment, by identifying the peaks of the local smooth pause times in the pause event curve graph, the concentrated positions of the cognitive stagnation of the reference object during the learning process of the teaching video can be captured.

[0031] Specifically, since pause events usually reflect the confusion, thinking or pause reactions generated by the reference object when understanding the teaching content, the local peaks of the pause times can characterize the time periods when the cognitive load in the teaching video significantly increases, that is, corresponding to potential knowledge point boundaries or teaching content switching points. Therefore, by defining the above local peaks as knowledge point boundaries, a behavior-driven structural division of the teaching video content can be achieved without relying on manual annotation of the corresponding knowledge point segments.

[0032] Furthermore, the recognition method further includes: S5. Obtain the reference behavior vector sequences of J reference objects according to the M knowledge point segments of the teaching video; Exemplarily, step S5 specifically further includes: S5-1. Mark the covered pause events on several time sub-axes involved in the knowledge point segment; S5-2. Using the original time serial numbers of the time sub-axes associated with the covered pause events as the time sequence position references, determine the start and end order of the knowledge point segment on the teaching time axis, and assign consecutive increasing advanced time serial numbers to each knowledge point segment based on this start and end order; Among them, the advanced time serial number is used to represent the time sequence position of the knowledge point segment in the teaching video; Exemplarily, the number rearrangement means: Determine the start order of each knowledge point segment on the teaching time axis according to the original time serial numbers of the time sub-axes to which the pause events belong; assign smaller advanced time serial numbers to the knowledge point segments with earlier start times, increasing in sequence until the advanced time serial numbers of each knowledge point segment are obtained; associate all the covered pause events of each knowledge point segment to the corresponding advanced time serial numbers; through unified number rearrangement, the knowledge point segments and their pause events maintain strict time sequence consistency in the new advanced time serial numbers.

[0033] S5-3. Construct a reference behavior vector indexed by the student identifier from the covered pause events; Among them, the features of the reference behavior vector include: student identifier, advanced time serial number, time stamp of each pause, pause duration, and total pause duration; Specifically, the reference behavior vector forms a behavior representation oriented to temporal distribution by integrating the timestamp sequence and duration sequence of all pause events of the reference object within the knowledge point segment; this vector can be used to reflect the cognitive resistance intensity and stagnant behavior distribution of the reference object in the knowledge point segment.

[0034] S5-4. Obtain the reference behavior vectors of the reference object for M knowledge point segments in the teaching video. S5-5. Sort the reference behavior vectors of the M knowledge point segments according to the advanced time serial numbers corresponding to the M knowledge point segments to generate a personalized reference behavior vector sequence. Among them, the reference behavior vector sequence is used to represent the personalized learning trajectory formed by the reference object on the teaching timeline. S5-6. Select J reference objects and obtain the reference behavior vector sequences of the J reference objects.

[0035] The reference behavior vector sequences of the J reference objects refer to the set of behavior vector sequences formed by arranging in sequence for each reference object according to the advanced time serial numbers of the knowledge point segments on the teaching timeline. The reference behavior vector sequence of each reference object reflects the change trajectory of the pause behavior on each knowledge point segment during the learning process of the teaching video, and can comprehensively describe the learning behavior pattern of the reference object under different knowledge contents.

[0036] Furthermore, the recognition method further includes: S6. Obtain the real-time behavior vector sequence of the object to be recognized and identify the most similar object of the object to be recognized from the J reference objects. Exemplarily, step S6 specifically further includes: S6-1. Obtain the real-time behavior vectors of the first Q knowledge point segments of the object to be recognized in the same teaching video and the first Q advanced time serial numbers. S6-2. Sort the real-time behavior vectors of the first Q knowledge point segments according to the first Q advanced time serial numbers to generate the real-time behavior vector sequence of the object to be recognized. S6-3. Determine the most similar object of the object to be recognized from the J reference objects according to the real-time behavior vector sequence of the object to be recognized.

[0037] The determination of the most similar object means that according to the real-time behavior vector sequence of the object to be recognized, similarity evaluations are respectively performed with the reference behavior vector sequences of the J reference objects, and the reference object with the highest overall similarity to the real-time behavior vector sequence is selected as the most similar object of the object to be recognized.

[0038] Among them, step S6-3 further includes: S6-3-1. Extract the first Q reference behavior vectors from the reference behavior vector sequences of J reference objects according to the advanced time serial numbers. S6-3-2. For each reference object, pair the real-time behavior vectors with the same advanced time serial number with the reference behavior vectors to form Q groups of time-series vector pairs. S6-3-3. Calculate the similarity of the time-series vector pairs in sequence according to the order of the advanced time serial numbers. If the similarity is less than the set threshold, terminate the matching of the current reference object and jump to the remaining reference objects; otherwise, calculate the similarity of the next group of time-series vector pairs. S6-3-4. If the similarities of the current Q groups of time-series vector pairs are all greater than the set threshold, mark this reference object as the most similar object. Specifically, the determination expression of the most similar object is:

[0039] Among them, S is the real-time behavior vector sequence, representing the behavior performance of the object to be recognized in the first Q knowledge point segments, which is sorted according to the advanced time serial numbers; T is the reference behavior vector sequence of the reference object, representing the behavior performance of a certain known reference object in the corresponding knowledge points in the same teaching video. ε is the similarity threshold, representing the preset minimum acceptable similarity, such as 0.85. vi is the i-th real-time behavior vector, representing the real-time learning behavior of the object to be recognized in the i-th knowledge point segment. ui is the i-th reference behavior vector, representing the reference learning behavior of the reference object in the i-th knowledge point segment. min(Sim) is the minimum similarity extraction, representing the similarity of the minimum time-series vector pair extracted in the first Q knowledge point segments. I(·) is the indicator function symbol, indicating that if the condition in the parentheses holds (that is, the minimum similarity ≥ ε), then output 1 (matching success), otherwise output 0 (matching failure). Match(·) is the matching function, representing the function output value for judging whether the reference object is similar to the object to be recognized in the first Q knowledge point segments.

[0040] In the learning behavior trajectory constructed based on the pause behavior of the teaching video, the real-time behavior sequence of the object to be recognized usually shows continuous cognitive response changes, rather than the isolated characteristics of a single knowledge point segment.

[0041] Therefore, in this embodiment, through the pairing and screening of continuous time-series vector pairs based on the order of the advanced time serial numbers, it is ensured that the real-time behavior sequence is highly consistent with the historical behavior trajectory of the reference object in the overall evolution trend.

[0042] Specifically, calculate the similarity of each pair of time series vectors corresponding to the advanced time serial numbers in sequence, and set a unified minimum similarity threshold. Only when all the Q group vector pairs currently meet the similarity threshold requirements, the corresponding reference object is determined as the most similar object of the object to be recognized. This method can not only avoid the interference of local pause anomalies (such as occasional thinking or interruption) on the overall matching, but also highlight the overall trajectory matching.

[0043] Furthermore, the recognition method further includes: S7. Determine the learning behavior evolution distribution of the object to be recognized according to the most similar object and its reference behavior vector sequence, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

[0044] Among them, step S7 specifically includes: S7-1. In the reference behavior vector sequences of J reference objects, select the reference behavior vector sequence of the most similar object; S7-2. From the reference behavior vector sequence of the most similar object, extract the pause timestamps and pause durations of the most similar object in each knowledge point segment, and calculate the learning difficulty of each knowledge point segment until the learning difficulties of M knowledge point segments are obtained. Exemplarily, for each knowledge point segment, calculate the learning difficulty based on the characteristics of the pause time.

[0045] Specifically, the learning difficulty is comprehensively determined according to the following two factors: The total number of pause events occurring within the knowledge point segment and the total duration of all pause events within the knowledge point segment.

[0046] Among them, the learning difficulty is proportional to the number of pauses, that is, the more the number of pauses, the greater the learning difficulty; at the same time, the learning difficulty is also proportional to the pause duration, that is, the longer the cumulative pause time, the greater the learning difficulty.

[0047] In the specific implementation process, certain weight coefficients can be assigned to the number of pauses and the pause duration respectively to reflect the relative influence degrees of different factors on the learning difficulty. The weight coefficients can be set according to experience or optimized and adjusted through a training process based on historical data.

[0048] Therefore, by performing weighted summation on the number of pauses and the pause duration, the learning difficulty value corresponding to each knowledge point segment can be obtained.

[0049] S7-3. Construct the learning behavior distribution of the most similar object according to the pause timestamps, pause durations, and learning difficulties of M knowledge point segments; Specifically, the learning behavior distribution is used to describe the learning feature change trajectory of the most similar object on each knowledge point segment of the teaching video.

[0050] Specifically, the learning behavior distribution is arranged in the chronological order of knowledge point segments based on the pause timestamps, pause durations, and learning difficulty data extracted from each knowledge point segment, forming a time series that reflects the evolution of cognitive load during the learning process. Through the above combination, the learning behavior distribution can completely represent the learning stagnation degree, the change trend of cognitive load, and the difficulty of mastering knowledge points shown by the most similar objects in different knowledge point segments, thus providing a data basis for personalized intervention strategies.

[0051] S7-4. Define the learning behavior distribution of the most similar object as the learning behavior evolution distribution of the object to be recognized, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

[0052] Specifically, the pre-trained learning intervention strategy refers to: An intervention strategy mapping relationship constructed in advance by classifying and training based on the historical learning behaviors of a large number of reference objects on different knowledge point segments according to the learning difficulty distribution, cognitive stagnation characteristics, and learning effectiveness indicators; Among them, the intervention strategies include but are not limited to: Learning resource recommendation (such as additional explanatory videos, auxiliary practice questions); Learning path adjustment (such as skipping mastered content, focusing on reviewing difficult points); Learning prompt generation (such as key reminders, question-answering guidance); Optimization of learning frequency and rhythm (such as increasing the review frequency, adjusting the push rhythm); During the system operation stage, the intervention strategy is dynamically matched according to the learning behavior evolution distribution of the object to be recognized.

[0053] In this embodiment, by collecting the pause events of the reference object during the playback of the teaching video on the teaching timeline, a pause event curve graph is generated, and the knowledge point segments are automatically divided based on the curve peak detection. By extracting the characteristics of the pause events associated with each knowledge point segment, the cognitive stagnation areas that occur during the learning process can be accurately captured, thus realizing fine-grained learning behavior modeling for teaching content and improving the recognition accuracy of changes in the learner's cognitive state.

[0054] Furthermore, in this embodiment, by collecting and processing pause event data in real time on the learning terminal side, the dependence on the central server is effectively reduced, and the response speed of learning behavior recognition and intervention decision-making is significantly improved. On this basis, by matching the reference object with the most similar learning trajectory to the object to be recognized in the reference object group, the learning evolution process of the object to be recognized on different knowledge point segments is dynamically inferred, and combined with the pre-trained learning intervention strategy, according to the learning difficulty distribution of the knowledge point segments, targeted learning resources and auxiliary measures are pushed, so as to realize the intelligent recommendation of personalized learning intervention.

[0055] The embodiment of the present invention also provides a learning behavior recognition system based on an edge computing architecture, which is used to implement the above method embodiment, and those that have been described will not be repeated here. The following terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0056] As Figure 4 shown, Figure 4 is a structural block diagram of a learning behavior recognition system based on an edge computing architecture of the present invention. The system includes: A teaching video acquisition module, configured to acquire a teaching video and its teaching timeline, and mark pause events occurring in the teaching video on the teaching timeline; A line chart generation module, configured to generate a pause event line chart of the teaching video according to the pause events occurring in the teaching video; A curve chart generation module, configured to smooth the lines of the pause event line chart to generate a pause event curve chart; A knowledge point segment determination module, configured to determine M knowledge point segments of the teaching video in the pause event curve chart; A reference sequence acquisition module, configured to acquire reference behavior vector sequences of J reference objects according to the M knowledge point segments of the teaching video; A most similar object recognition module, configured to acquire a real-time behavior vector sequence of the object to be recognized, and recognize the most similar object of the object to be recognized from the J reference objects; A learning behavior intervention module, configured to determine the learning behavior evolution distribution of the object to be recognized according to the most similar object and its reference behavior vector sequence, and match a pre-trained learning intervention strategy based on the learning behavior evolution distribution.

[0057] In the above system, through the teaching video acquisition module, the teaching video and its pause events are obtained; through the line graph generation module, a line graph of pause events is generated; through the curve graph generation module, a curve graph of pause events is generated; through the knowledge point segment determination module, the knowledge point segments are determined; through the reference sequence acquisition module, the reference behavior vector sequence is obtained; through the most similar object recognition module, the most similar object is recognized; and through the learning behavior intervention module, the learning intervention strategy is matched; thereby solving the problem of difficult to accurately identify the cognitive stagnation phenomenon that occurs at the specific knowledge point position of the learner.

[0058] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means.

[0059] The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more collections of available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, DVD ), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0060] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0061] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. A learning behavior recognition method based on edge computing, characterized in that, The described behavior recognition method includes: S1. Obtain the teaching video and its teaching timeline, and mark the pause events that occur in the teaching video on the teaching timeline; S2. Generate a pause event line graph of the teaching video according to the pause events that occur in the teaching video; S3. Smooth the line of the pause event line graph to generate a pause event curve graph; S4. In the pause event curve graph, determine M knowledge point segments of the teaching video; S5. According to the M knowledge point segments of the teaching video, obtain the reference behavior vector sequences of J reference objects; S6. Obtain the real-time behavior vector sequence of the object to be recognized, and identify the most similar object of the object to be recognized from the J reference objects; S7. According to the most similar object and its reference behavior vector sequence, determine the learning behavior evolution distribution of the object to be recognized, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

2. The learning behavior recognition method based on edge computing according to claim 1, wherein The generating a pause event line graph of the teaching video according to the pause events that occur in the teaching video S2-1. Divide the teaching timeline into N time sub-axes of the same length, and assign the original time serial numbers based on the time sequence to each time sub-axis; S2-2. Obtain several pause events of the teaching video within each time sub-axis; Among them, each pause event is associated with its original time serial number, student identifier, the time stamp of this pause, and the pause duration to form a set of pause events within the time sub-axis; S2-3. Use the teaching timeline as the X-axis and the number of pause events as the Y-axis to construct an initial coordinate graph; S2-4. In the initial coordinate graph, mark the number of pause events of the teaching video within each time sub-axis to generate N pause points; S2-5. Connect any adjacent pause points to generate the pause event line graph of the teaching video.

3. The learning behavior recognition method based on edge computing according to claim 1, wherein Smoothing the line of the pause event line graph to generate a pause event curve graph includes: S3-1. Mark three arbitrarily adjacent time sub-axes; S3-2. Obtain the original time serial numbers of the three arbitrarily adjacent time sub-axes, and define the time sub-axis corresponding to the smallest original time serial number as the anchor axis; S3-3. Obtain the total number of pause events of the three arbitrarily adjacent time sub-axes; S3-4. Calculate the average number of pause events according to the total number of pause events of the three arbitrarily adjacent time sub-axes; S3-5. Define the average number of pause events as the smoothed number of pause events of the anchor axis; S3-6. Repeat S3-1 to S3-5 until the smoothed number of pause events of each time sub-axis is obtained; S3-7. Mark the smoothed number of pause events of each time sub-axis to generate N smoothed pause points; S3-8. Connect any adjacent smoothed pause points to generate the pause event curve graph of the teaching video.

4. A learning behavior recognition method based on edge computing according to claim 1, characterized in that, In the pause event curve graph, determining M knowledge point segments of the teaching video includes: S4-1. Mark the knowledge point peaks in the K smoothed pause point sets in the pause event curve graph; S4-2. Extend the dividing lines along the K knowledge point peaks to the teaching timeline to divide the teaching video into M knowledge point segments; where M = K + 1.

5. A learning behavior recognition method based on edge computing according to claim 4, characterized in that, Marking the knowledge point peaks in the K smoothed pause point sets in the pause event curve graph includes: S4-1-1. In the pause event curve graph, select the smoothed pause count corresponding to each time sub-axis; S4-1-2. Determine whether the smoothed pause count corresponding to each time sub-axis is greater than the smoothed pause counts of its two adjacent time sub-axes before and after; S4-1-3. If it is greater, mark the smoothed pause point as a candidate peak in the pause event curve graph; S4-1-4. Repeat S4-1-2 to S4-1-3 until all candidate peaks in the pause event curve graph are marked; S4-1-5. Calculate the time distance between adjacent candidate peaks among all candidate peaks; S4-1-6. If the time distance is less than the set minimum threshold, mark the larger peak among the adjacent candidate peaks as a knowledge point peak, otherwise, mark both adjacent candidate peaks as knowledge point peaks until K knowledge point peaks are obtained.

6. The learning behavior recognition method based on edge computing according to claim 1, characterized in that, Based on the M knowledge point segments of the teaching video, obtain the reference behavior vector sequences of J reference objects, including: S5-1. Mark the covered pause events on several time sub-axes involved in the knowledge point segment; S5-2. Use the original time serial numbers of the associated time sub-axes in the covered pause events as the time sequence position reference, determine the start and end order of the knowledge point segment on the teaching time axis, and assign consecutive increasing advanced time serial numbers to each knowledge point segment based on this start and end order; Among them, the advanced time serial number is used to represent the time sequence position of the knowledge point segment in the teaching video; S5-3. Construct a reference behavior vector indexed by student identification from the covered pause events; Among them, the features of the reference behavior vector include: student identification, advanced time serial number, time stamp of each pause, pause duration, and total pause duration; S5-4. Obtain the reference behavior vectors of the reference objects for the M knowledge point segments in the teaching video; S5-5. Sort the reference behavior vectors of the M knowledge point segments according to the advanced time serial numbers corresponding to the M knowledge point segments to generate a personalized reference behavior vector sequence; Among them, the reference behavior vector sequence is used to represent the personalized learning trajectory formed by the reference object on the teaching time axis; S5-6. Select J reference objects and obtain the reference behavior vector sequences of the J reference objects.

7. A learning behavior recognition method based on edge computing according to claim 1, characterized in that Obtain the real-time behavior vector sequence of the object to be recognized and identify the most similar object of the object to be recognized from J reference objects, including: S6-1. Obtain the real-time behavior vectors and the first Q advanced time serial numbers of the first Q knowledge point segments of the object to be recognized in the same teaching video; S6-2. Sort the real-time behavior vectors of the first Q knowledge point segments according to the first Q advanced time serial numbers to generate the real-time behavior vector sequence of the object to be recognized; S6-3. Determine the most similar object of the object to be recognized from J reference objects according to the real-time behavior vector sequence of the object to be recognized.

8. A learning behavior recognition method based on edge computing according to claim 7, characterized in that, Determine the most similar object of the object to be recognized from J reference objects according to the real-time behavior vector sequence of the object to be recognized, including: S6-3-1. Extract the first Q reference behavior vectors from the reference behavior vector sequences of J reference objects according to the advanced time serial number; S6-3-2. For each reference object, pair the real-time behavior vectors with the same advanced time serial number with the reference behavior vectors to form Q groups of time-series vector pairs; S6-3-3. Calculate the similarity of the time-series vector pairs in sequence according to the order of the advanced time serial numbers; If the similarity is less than the set threshold, terminate the matching of the current reference object and jump to the remaining reference objects; otherwise, calculate the similarity of the next group of time-series vector pairs; S6-3-4. If the similarities of all the Q groups of time-series vector pairs are greater than the set threshold, mark this reference object as the most similar object.

9. A learning behavior recognition method based on edge computing according to claim 1, characterized in that Based on the most similar object and its reference behavior vector sequence, determine the learning behavior evolution distribution of the object to be recognized, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution, including: S7-1. In the reference behavior vector sequences of J reference objects, select the reference behavior vector sequence of the most similar object; S7-2. From the reference behavior vector sequence of the most similar object, extract the pause timestamps and pause durations of the most similar object in each knowledge point segment, and calculate the learning difficulty of each knowledge point segment until the learning difficulties of M knowledge point segments are obtained; S7-3. Construct the learning behavior distribution of the most similar object according to the pause timestamps, pause durations and learning difficulties of the M knowledge point segments; S7-4. Define the learning behavior distribution of the most similar object as the learning behavior evolution distribution of the object to be recognized, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

10. A learning behavior recognition system based on an edge computing architecture, characterized in that, Including: A teaching video acquisition module, which is used to acquire the teaching video and its teaching timeline, and mark the pause events occurring in the teaching video on the teaching timeline; A line chart generation module, which is used to generate a pause event line chart of the teaching video according to the pause events occurring in the teaching video; A curve chart generation module, which is used to smooth the line of the pause event line chart to generate a pause event curve chart; A knowledge point segment determination module, which is used to determine M knowledge point segments of the teaching video in the pause event curve chart; A reference sequence acquisition module, which is used to acquire the reference behavior vector sequences of J reference objects according to the M knowledge point segments of the teaching video; A most similar object recognition module, which is used to acquire the real-time behavior vector sequence of the object to be recognized and recognize the most similar object of the object to be recognized from J reference objects; A learning behavior intervention module, which is used to determine the learning behavior evolution distribution of the object to be recognized according to the most similar object and its reference behavior vector sequence, and match the pre-trained learning intervention strategy based on the learning behavior evolution distribution.

Images