Resource calling method and device based on educational data, medium and program product

Abstract

The invention provides a resource calling method and device based on education data, equipment, a medium and a program product, and the method comprises the steps: generating behavior event information corresponding to the education data, and transmitting the behavior event information and the education data to a Kafka partition corresponding to a user ID; according to the behavior event information and the education data in the Kafka partition, space-time normalization processing and similarity analysis are carried out, and user standardized event information is generated; performing Granger causal analysis according to the standardized event information of the user to generate a causal graph; and under the condition that the resource scheduling is determined to be triggered according to the causal graph, generating resource collaborative scheduling information by utilizing the resource chain template, and carrying out system resource collaborative calling. According to the method and the device, the causal identification accuracy of the multi-modal education data can be improved, the false triggering rate of resource calling is reduced, the resource pushing precision is improved, the resource collaboration efficiency is improved, the processing delay of the multi-modal data can be reduced, and the rule adaptability is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a resource calling method and device based on education data, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the expansion of the scale of online education platforms, education data has developed from single text courseware to multi-modal data such as video live broadcast, audio explanation, interactive test, learning behavior log, etc. At present, there are two schemes for resource calling: a static recommendation method based on multi-modal feature fusion and a rule-driven resource allocation method for live broadcast scenarios. The static recommendation method collects multi-modal data such as video viewing behavior, courseware interaction log, test answer record, etc., uses multi-modal feature extraction technology, inputs the spliced multi-modal feature vector into a deep neural network to generate a recommendation list, and triggers the recommendation service when the user learning time exceeds a preset threshold. The rule-driven resource allocation method monitors the user concurrency and bandwidth occupancy rate in real time for high-concurrency live broadcast scenarios, and realizes conditional expansion of server resources through a pre-defined rule engine. However, the static recommendation method relies on statistical correlation rather than causal reasoning, and cannot distinguish between behavior manifestations and real learning intentions. The rule-driven resource allocation method can only deal with explicit causality and lacks the ability to mine implicit causal chains. Therefore, the static recommendation method and the rule-driven resource allocation method have low accuracy in causal identification of multi-modal education data, resulting in high false triggering rate of resource calling, and problems such as resource mismatch and resource repeated overhead. SUMMARY

[0003] The present disclosure provides a resource calling method and device based on education data, a computer readable storage medium and a computer program product.

[0004] According to a first aspect of the present disclosure, a resource calling method based on education data is provided, comprising: performing metadata extraction, space-time alignment processing and quality detection processing on received education data to generate behavior event information corresponding to the education data; wherein the education data includes video, audio, course content and behavior log; the behavior log includes event type and event time; the behavior event information includes user ID and timestamp; according to the user ID, the behavior event information and the corresponding education data are sent to a Kafka partition corresponding to the user ID, wherein the user ID and the Kafka partition correspond one by one; according to the behavior event information and the education data in the Kafka partition, space-time normalization processing and similarity analysis are performed to generate user standardized event information; wherein the user standardized event information includes standardized event tuple, confidence score and evidence trace information, the standardized event tuple includes user ID, event type, knowledge node and timestamp; Granger causality analysis is performed according to the user standardized event information to generate a causal diagram; in the case of determining to trigger resource scheduling according to the causal diagram, resource collaborative scheduling information is generated by using a resource chain template according to the causal diagram and the user standardized event information; system resource collaborative calling processing is performed based on the resource collaborative scheduling information, and system resource state is monitored.

[0005] Optionally, the quality detection processing includes quality grading processing; the metadata extraction, space-time alignment processing and quality detection processing on the received education data to generate the behavior event information corresponding to the education data includes: performing metadata extraction on the education data to obtain metadata corresponding to the education data, the metadata including the timestamp, coordinate information, user ID and data feature information; performing the space-time alignment processing on the education data, the space-time alignment processing including coordinate conversion processing, cross-device clock synchronization processing based on PTP protocol and resampling compensation processing; performing detection processing on the education data based on QoS strategy; generating the behavior event information in JSON format, wherein the behavior event information further includes global event ID, data type label, data information and service level.

[0006] Optionally, the generating user standardized event information according to the behavior event information in the Kafka partition and the education data comprises: performing slice processing on the video and the audio to generate video slices and audio slices; determining corresponding knowledge graph nodes in a knowledge graph according to the course content; performing spatial conversion processing on the video slices by using a space-time conversion matrix; obtaining audio text corresponding to the audio slices by using a Whisper speech recognition model; determining video slices, course content, and audio text associated with an event type according to a timestamp of the behavior event information; obtaining a first cosine similarity between the video slices and the course content associated with the event type by using a CLIP image-text alignment model; obtaining a second cosine similarity between the audio text and the course content associated with the event type by using a BERT audio-text alignment model; generating the standardized event tuple, the confidence score, and the evidence trace information in a case where the first cosine similarity is greater than a dynamic threshold and the corresponding second cosine similarity is greater than an audio-text threshold; wherein the standardized event tuple comprises a user ID, an event type, a knowledge graph node, and a timestamp; and the evidence trace information comprises a path for storing associated video frames and / or audio slices.

[0007] Optionally, at least one of the dynamic threshold and the audio-text threshold is adjusted according to a reference threshold, a user type factor, a content type factor, and a delay compensation factor; and the confidence score is determined based on a basic confidence and processing delay information.

[0008] Optionally, the generating a causal graph according to the user standardized event information comprises: generating a feature vector based on event types in the user standardized event information, and constructing an event sequence matrix according to the feature vector; performing Granger test on each pair of event types in all event types by using a vector autoregression model according to the event sequence matrix, to determine a causal probability between the each pair of event types; determining an occurrence probability between the each pair of event types according to the causal probability between the each pair of event types; and generating the causal graph according to the event types and the occurrence probability between the each pair of event types; wherein a node in the causal graph represents an event type, and a weighted edge between nodes in the causal graph represents an occurrence probability between the nodes.

[0009] Optionally, the determining the occurrence probability between each pair of event types according to the causal probability between the pair of event types comprises: determining an event occurrence time corresponding to the pair of event types, and determining a time difference between the event occurrence time and a current time; determining a time-decay factor according to a decay rate and the time difference; and determining a product of the causal probability and the time-decay factor as the occurrence probability between the pair of event types.

[0010] Optionally, after the causal graph is generated, when it is determined that a causal probability between a new pair of event types is greater than a causal rule threshold, the new pair of event types is added to the causal graph; a new occurrence probability of a weighted edge in the causal graph is calculated, and an existing occurrence probability of the weighted edge in the causal graph is updated.

[0011] Optionally, in a case where it is determined that the occurrence probability of the weighted edge in the causal graph is greater than the scheduling trigger threshold, it is determined to perform resource scheduling.

[0012] Optionally, a reward function is constructed according to a delay rate, a resource utilization rate, and a false trigger rate; an adjustment action is determined by using a rule optimization model according to collected first system running state information, wherein the first system running state information includes the delay rate, the resource utilization rate, and the false trigger rate; the adjustment action includes adjusting a scheduling trigger threshold and adjusting a proportion of resources that need to be scheduled; after the adjustment action is executed, a reward value is calculated by using the reward function according to newly collected second system running state information, wherein the second system running state information includes the delay rate, the resource utilization rate, and the false trigger rate; training samples are generated and put into an experience pool based on the first system running state information, the adjustment action, the reward value, and the second system running state information; in a case where a model training condition is met, the training samples in the experience pool are used to implement parameter optimization of the rule optimization model by policy gradient update, so as to train the rule optimization model, with the reward value as an optimization target.

[0013] Optionally, the trained rule optimization model is used to adjust the scheduling trigger threshold and / or adjust the proportion of resources that need to be scheduled; wherein the resources that need to be scheduled include GPU resources and CDN resources.

[0014] Optionally, the generating, according to the causal graph and the user standardized event information, resource coordination scheduling information by using a resource chain template comprises: determining, according to the causal graph, the user standardized event information, and the proportion of resources that need to be scheduled, an event type and a scheduling parameter that need to be scheduled by using a trained analysis model; determining resource requirement information based on the scheduling parameter; wherein the resource requirement information comprises computing resource requirement information, storage resource requirement information, and network resource requirement information; determining a resource chain template corresponding to the event type that need to be scheduled, wherein the resource chain template comprises a YAML template; and generating the instantiated template as the resource coordination scheduling information according to the resource requirement information and the resource chain template.

[0015] Optionally, the system resource coordination calling processing based on the resource coordination scheduling information comprises: calling, according to the resource coordination scheduling information, an interface to perform resource scheduling processing resources in parallel to make resources work coordinately; wherein the interface comprises a K8S API for expanding GPU Pods, a CDN preheating interface, and an interface for configuring an SDN flow table.

[0016] According to a second aspect of the present disclosure, a resource calling device based on education data is provided, comprising a memory and a processor coupled to the memory, the processor being configured to execute the method as described above based on instructions stored in the memory.

[0017] According to a third aspect of the present disclosure, a computer readable storage medium is provided, which stores computer instructions, and the computer instructions are executed by a processor to execute the method as described above.

[0018] According to a fourth aspect of the present disclosure, a computer program product is provided, which stores computer instructions, and the computer instructions are executed by a processor to execute the method as described above.

[0019] The resource calling method, device, computer readable storage medium, and computer program product based on education data provided by the present disclosure can improve the accuracy of causal recognition of multi-modal education data, reduce the false triggering rate of resource calling, improve the accuracy of resource pushing, and improve the efficiency of dynamic resource coordination; can reduce the processing delay of multi-modal data, enhance the adaptability of rules, reduce the system response delay, reduce the platform operation cost, and can be applied to diversified application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. The drawings provided below are for illustrative purposes and are included to further convey the principles of the present disclosure and are not intended to limit the scope of the present disclosure. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. The drawings provided below are for illustrative purposes and are included to further convey the principles of the present disclosure and are not intended to limit the scope of the present disclosure. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. In the drawings provided below, like reference numerals will be used to represent like parts throughout the several views.

[0021] Figure 1 Flowchart for generating behavior event information for some embodiments of the resource calling method based on education data according to the present disclosure;

[0022] Figure 2 Flowchart for generating behavior event information for some embodiments of the resource calling method based on education data according to the present disclosure;

[0023] Figure 3 Flowchart for generating user standardized event information for some embodiments of the resource calling method based on education data according to the present disclosure;

[0024] Figure 4 Flowchart for determining confidence score for some embodiments of the resource calling method based on education data according to the present disclosure;

[0025] Figure 5 Flowchart for generating causal diagram for some embodiments of the resource calling method based on education data according to the present disclosure;

[0026] Figure 6 Flowchart for determining occurrence probability for some embodiments of the resource calling method based on education data according to the present disclosure;

[0027] Figure 7 Causal diagram for some embodiments of the resource calling method based on education data according to the present disclosure;

[0028] Figure 8 Flowchart for updating causal diagram for some embodiments of the resource calling method based on education data according to the present disclosure;

[0029] Figure 9 Flowchart for training rule optimization model for some embodiments of the resource calling method based on education data according to the present disclosure;

[0030] Figure 10 Flowchart for generating resource collaborative scheduling information for some embodiments of the resource calling method based on education data according to the present disclosure;

[0031] Figure 11 A block schematic of some embodiments of an educational data based resource invocation apparatus according to the present disclosure. DETAILED DESCRIPTION

[0032] Exemplary embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. In the description, all the features of the embodiments are not described in order to make the description clear and brief. It should be appreciated that many implementation-specific decisions can have to be made in order to develop a specific implementation, which will depend on specific design constraints imposed by a particular application or constraints imposed by a particular implementation, which can vary from one implementation to another. Moreover, it should be appreciated that depending on the implementation and how it is implemented, the development work (e.g. the steps taken to complete an implementation) can have been very complex and time-consuming, but such development work would merely be a routine task for those skilled in the art having the benefit of this disclosure.

[0033] It should be noted that the relative arrangement of the components and steps, numerical expressions, and numerical values set forth in the embodiments are not intended to limit the scope of the present disclosure unless otherwise specifically stated.

[0034] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are merely used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor indicate their logical order.

[0035] It should also be understood that in the embodiments of the present disclosure, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0036] It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, unless specifically limited or unless the context clearly indicates otherwise, it can be understood as one or more in general.

[0037] In addition, the term "and / or" in the present disclosure is merely a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects.

[0038] It should also be understood that the description of various embodiments of the present disclosure focuses on the differences between the various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0039] Meanwhile, it should be understood that, for the convenience of description, the size of each part shown in the drawings is not drawn in accordance with the actual proportional relationship.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application or uses.

[0041] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification.

[0042] It should be noted that like reference numerals and letters refer to like items throughout the several views, and thus a discussion of the same item in one view does not require discussion of the same item in the other views.

[0043] Furthermore, in the interest of not obscuring the disclosure, only processing steps and / or apparatus structures pertinent to the heavily relied upon aspects of the disclosure in accordance with at least one embodiment are shown in the drawings, and other details deemed by the patentees to be less pertinent are either omitted or only briefly described. It should also be noted that like reference numerals and letters in the accompanying drawings indicate like items, and thus a discussion of the same item in one drawing does not require a discussion of the same item in the other drawings.

[0044] In the related art known to the inventors, a static recommendation method based on multi-modal feature fusion adopts a multi-modal feature fusion architecture to implement education resource recommendation, relies on statistical correlation rather than causal reasoning, and cannot distinguish between behavior manifestations and real learning intentions. For example, a user repeatedly pauses a video due to network congestion, which is misjudged by the system as "knowledge point confusion" to trigger the push of a wrong question analysis courseware, but in fact the network detection tool needs to be called.

[0045] A rule-driven resource allocation method for live streaming scenarios is adopted. Although the rule engine responds quickly, it can only handle explicit causes such as "teacher opens Q&A -> student number increases", lacks the ability to mine implicit causal chains, and requires a large amount of manually maintained rule data with insufficient coverage.

[0046] The existing static recommendation method and rule-driven resource allocation method have failed to build a technical closed loop of "multi-modal data -> causal event identification -> resource dynamic scheduling": the static recommendation method decouples recommendation and resource, resulting in a split between feature calculation and resource calling under high concurrency; the rule-driven resource allocation method can achieve fast resource allocation, but the allocation strategy is disconnected from the causal learning behavior, ultimately forming a defect chain of "mis-triggering -> high latency -> resource mismatch". Therefore, the existing methods have low accuracy in causal identification of multi-modal education data, resulting in high mis-triggering rate of resource calling, and prone to problems such as resource mismatch and resource duplication.

[0047] Figure 1 A flowchart of some embodiments of the resource calling method based on education data according to the disclosure is shown in FIG. 1. Figure 1shown:

[0048] In step S101, metadata extraction, space-time alignment processing and quality detection processing are performed on the received education data to generate behavior event information corresponding to the education data.

[0049] The education data is multi-modal education data with space-time characteristics and semantic features. The education data includes video, audio, course content, behavior log and other data. The behavior log includes event type and event time information. The behavior event information includes user ID, timestamp and other information.

[0050] For example, in the case of a student of online education platform A participating in a math online live class, the received education data includes: the explanation video pushed by the teacher through the live device, the audio of the teacher's explanation, the structured course content opened by the student using the terminal, and the behavior log of the student during the class. The behavior log of the student during the class includes event type and event time information. The event type can include pausing the video, replaying, downloading request, etc. The event time is the time corresponding to pausing the video, replaying, downloading request, etc.

[0051] The video stream can be extracted by supporting dynamic code rate adjustment and performing key frame extraction per second. The audio stream is transmitted by WebSocket long connection, and the sampling rate is standardized and real-time environmental noise reduction is performed. The course content is accessed through RESTful API to complete content structural decomposition and version control. The log data is collected by the agent to realize incremental log capture and sensitive data desensitization processing driven by regular expressions to form complete multi-modal access capability.

[0052] Various methods can be used to receive education data. For example, video stream is received using RTMP / WebRTC protocol, audio stream is received based on WebSocket protocol, course content is received through RESTful API, and behavior log is received by Filebeat agent. Filebeat is a lightweight log collection tool.

[0053] In step S102, the behavior event information and corresponding education data are sent to the Kafka partition corresponding to the user ID according to the user ID.

[0054] The behavior event bus can be used as a transfer hub for behavior event information, and the behavior event bus can be implemented using a Kafka message queue. The Kafka message queue can implement efficient and reliable message transmission in a distributed system, and supports large-scale real-time data publishing and subscription. Kafka partition is the smallest unit of parallelism of Kafka, and a Kafka partition can be configured for each user, so that the user ID corresponds to the Kafka partition one by one; each Kafka partition can be independently stored in different server nodes of the Kafka cluster, realizing distributed storage of messages.

[0055] Various methods can be used to obtain behavior event information corresponding to each user in the behavior event information. For example, based on the user ID in the behavior event information, the behavior event information corresponding to each user is obtained in the behavior event information; by performing hash calculation on the user ID, the behavior event information corresponding to each user is sent to the Kafka partition corresponding to each user, ensuring that the events of the same user can be processed in order. A 5-second TTL (Time To Live) can be set for the message, and the behavior event information is automatically discarded when it expires, to ensure the real-time performance of data processing.

[0056] In step S103, the behavior event information in the Kafka partition and the education data are used to perform spatio-temporal normalization processing and similarity analysis, and user standardized event information is generated. The user standardized event information includes standardized event tuples, confidence scores, and evidence trace information, and the standardized event tuples include user ID, event type, knowledge node, and timestamp.

[0057] In step S104, Granger causality analysis is performed on the user standardized event information to generate a causal graph. The Granger causality analysis method is an analysis method used to determine the statistical prediction relationship between two variables in econometrics and time series analysis.

[0058] In step S105, when it is determined that the resource scheduling is triggered according to the causal graph, resource chain templates are used to generate resource collaborative scheduling information according to the causal graph and the user standardized event information.

[0059] In step S106, system resource collaborative calling processing is performed based on the resource collaborative scheduling information, and the system resource state is monitored.

[0060] The resource chain template is used to generate resource cooperative scheduling information, and a resource relationship graph is established according to the resource cooperative scheduling information and in dependence on analysis. The resource relationship graph is a graph used to describe the dependence relationship and topological structure among heterogeneous resources such as computing, storage and network, and can provide a basis for dynamic cooperative scheduling of resources, ensure that resource scheduling is executed in sequence and efficiently, and realize cooperative linkage of computing, storage and network resources.

[0061] The system resources include GPU computing resources, distributed storage systems, CDN content distribution networks and underlying network facilities and the like. Prometheus can be integrated to monitor the state of the system resources, and closed-loop monitoring of operation indicators is realized. Prometheus is a system monitoring and alarm tool.

[0062] The resource calling method based on education data provided by the present disclosure can improve the accuracy of causal identification of multi-modal education data, reduce the false triggering rate of resource calling, improve the accuracy of resource pushing, and improve the efficiency of dynamic resource cooperation. It can reduce the processing delay of multi-modal data and enhance the adaptability of rules.

[0063] Figure 2 For the flowchart of generating behavior event information according to some embodiments of the resource calling method based on education data of the present disclosure, quality detection processing includes quality grading processing and the like, as shown in Figure 2

[0064] In step S201, metadata extraction is performed on the education data to obtain metadata corresponding to the education data.

[0065] The metadata includes timestamp, coordinate information, user ID and data feature information, etc. The metadata extraction engine can be used to perform metadata extraction and other processing on the education data. The metadata extraction engine automatically captures metadata, including NTP synchronized millisecond timestamp (for example, the timestamp of a video pause event is millisecond level), fusion GPS / WiFi fingerprint space coordinate (for example, teaching building n floor n classroom), user ID based on biometric features, and data feature information, etc. The data feature information includes video resolution, audio signal-to-noise ratio and other features.

[0066] The metadata extraction engine can realize millisecond timestamp synchronization through NTP protocol, obtain three-dimensional space coordinates using fusion fingerprint positioning technology, and integrate gyroscope data to analyze device posture; the metadata extraction engine can establish a biometric feature binding mechanism through a user identity system, realize cross-device ID mapping and anonymous user identification generation, and automatically mark data feature attributes, including video resolution and frame rate, audio signal-to-noise ratio, courseware text complexity index, etc.

[0067] ​In step S202, the education data is subjected to spatio-temporal alignment processing, including coordinate conversion processing, cross-device clock synchronization processing based on the PTP protocol, and resampling compensation processing.

[0068] The education data can be subjected to spatio-temporal alignment processing using various methods. For example, the education data can be subjected to spatio-temporal alignment processing using a spatio-temporal alignment processor, including coordinate conversion from world coordinates to relative coordinates, cross-device clock synchronization based on the PTP protocol (ensuring that the audio-video alignment error is less than 50 ms), and resampling compensation of non-uniformly sampled data.

[0069] The spatio-temporal alignment processor can convert the world coordinate system collected by the multi-source device into a relative coordinate system in the spatial dimension, for example, converting the teacher's camera world coordinates (x=10.2, y=5.3) and the student's mobile phone screen coordinates (x=0.5, y=0.3) into relative coordinates with the "matrix operation" knowledge point as the origin, eliminating the multi-device perspective difference.

[0070] The spatio-temporal alignment processor can eliminate the perspective difference through a multi-camera view fusion algorithm, achieve cross-device microsecond-level synchronization in the time dimension using the PTP precise clock protocol, and control the synchronization error in combination with audio-video lip alignment technology. For example, the PTP protocol is used to calibrate the teacher's microphone and camera clock, ensuring that the lip alignment error between "explanation voice" and "blackboard action" is less than 30 ms (much lower than the 50 ms threshold). For resampling compensation processing, the spatio-temporal alignment processor can implement a fixed interval resampling strategy for non-uniformly sampled data and use a linear interpolation algorithm to compensate for missing frames, ensuring the consistency of multi-modal data in the spatio-temporal dimension. For example, for the 2 frames of video lost due to the student's network lag, a transition frame is generated through a linear interpolation algorithm to ensure the continuity of the picture.

[0071] In step S203, the education data is subjected to detection processing based on a QoS strategy.

[0072] A three-level QoS strategy can be implemented for the education data: real-time audio-video stream requires a delay of less than 100 ms (QoS1), courseware update tolerates a delay of less than 1 s (QoS2), and historical log processing has a delay of less than 5 s (QoS3), while providing data packet retransmission, damaged data isolation, and real-time quality alarm mechanisms.

[0073] Based on the QoS strategy, the education data can be subjected to three-level detection processing: integrity verification through key frame continuity detection to ensure video stream integrity, waveform analysis to verify audio continuity; based on the QoS grading strategy, the data stream is divided into three service levels: real-time audio-video, quasi-real-time courseware update, and historical log processing; the abnormal processing mechanism includes data packet retransmission, damaged data isolation sandbox, and quality degradation real-time alarm based on a sliding window, constructing an end-to-end quality protection system.

[0074] For example, integrity check is performed: it is detected that the 120th frame of the video stream is lost, a key frame continuity alarm is triggered, and automatic retransmission of the fragment is requested; through audio waveform analysis, it is confirmed that the teacher's explanation has no pause (continuity meets the standard); abnormal processing: a damaged courseware fragment (which cannot be parsed) is sent, and it is automatically isolated to the sandbox.

[0075] In step S204, behavior event information in JSON format is generated.

[0076] The behavior event information includes global event ID, timestamp, user ID, data type label, data information, and the like; the data information can be detailed payload information, and the detailed payload information is core information of video stream, audio stream, behavior log, and the like recorded in the standardized JSON event format, for example, for the video stream, the detailed payload information can include knowledge points corresponding to key frames, video picture content description, video feature parameters, and playing time period; for the audio stream, the detailed payload information includes voice translation text fragments and audio features; for the behavior log, the detailed payload information includes specific operation details, such as "the URL of the downloaded courseware is A", "the video pause duration is 8 seconds", "the operation associated resource identifier", and the like.

[0077] Through experiments, it is verified that the standardized JSON event format can support a peak throughput of 50,000 events per second, and ensure that the end-to-end processing delay value is less than 80 ms; through a data loss rate of <0.001% and a fault switching capability of 200 ms level, service continuity can be ensured.

[0078] In some embodiments, the cross-modal alignment engine can be used to generate standardized event tuples through spatio-temporal normalization technology, CLIP model calculation graph-text similarity, and the like, to solve the problem of spatio-temporal asynchrony of multi-modal data. Figure 3 A flowchart for generating user standardized event information according to some embodiments of the resource calling method based on educational data according to the present disclosure is shown in FIG. 1. Figure 3 As shown in FIG. 1.

[0079] In step S301, the video and audio are sliced to generate video slices and audio slices. For example, the video and audio stream can be sliced according to a fixed window of 5 seconds to generate video slices and audio slices.

[0080] In step S302, according to the courseware content, the corresponding knowledge graph node in the knowledge graph is determined.

[0081] The knowledge graph can be established in advance. The knowledge graph is a structured semantic network taking entities as nodes and the relationships between entities as edges, and is used to clearly describe various things and their interrelations. In the knowledge graph, the node is the basic unit of the graph, represents an identifiable entity or concept, and is the core element of the knowledge graph that carries information.

[0082] The content of the courseware (such as the content of PPT, PDF, etc.) is extracted to obtain courseware elements, including text paragraphs, titles, charts, formulas, chapter titles, etc. The hash value of the courseware element can be input into the trained large model to calculate the similarity between the courseware element and the knowledge graph node. If the similarity is greater than a threshold, the courseware element is associated with the corresponding node in the knowledge graph, and the corresponding knowledge graph node is determined in the knowledge graph.

[0083] In step S303, the video slice is processed by spatial conversion using a space-time conversion matrix.

[0084] The space-time coordinate transformation matrix can be set as:

[0085]

[0086] For the video slice, a homography matrix H can be used to realize the spatial conversion of the camera coordinates to the virtual canvas ([x′y′1]=H[xy1]), convert to a unified virtual coordinate system, and eliminate the spatial misplacement caused by different viewing angles and device specificity.

[0087] Through the space-time normalization processing of steps 301-303, the coordinate system of the multi-modal education data can be unified, and the coordinate system can be unified as user ID, UTC timestamp, virtual coordinates, etc. Drift compensation can be performed, that is, the clock is calibrated every hour with a ±200ms window adjustment.

[0088] In step S304, the audio text corresponding to the audio slice is obtained by using the Whisper speech recognition model.

[0089] MFCC (Mel Frequency Cepstral Coefficients) is an acoustic feature in speech signal processing, which is used to convert the time domain signal of speech into a frequency domain feature that can better reflect the human auditory perception characteristics. Whisper is a general speech recognition model, and the audio text corresponding to the audio slice can be obtained by using the Whisper speech recognition model.

[0090] In step S305, the video slice, courseware content, and audio text associated with the event type are determined according to the timestamp of the behavior event information.

[0091] In some embodiments, the millisecond-level time reference t can be established by PTP clock synchronization align = max(t v ,t a ,t s )+n x 5000). The association rule of the educational data can be set, for example, the association rule includes that the difference between the timestamps in the behavior event information of different educational data is less than a time difference threshold.

[0092] The video clip, course content, audio text associated with the event type in the behavior log can be determined according to the timestamps of the behavior event information corresponding to the video, audio, course content, and behavior log using the association rule.

[0093] Step S306, obtaining the first cosine similarity between the video clip and the course content associated with the event type using the CLIP text-image alignment model;

[0094] In some embodiments, the text-image alignment model can be various models, such as the CLIP text-image alignment model and the like. The CLIP (Contrastive Language-Image Pre-Training) text-image alignment model is a multi-modal pre-training neural network that learns effective and scalable methods from natural language supervision, which can learn the alignment relationship between images and texts.

[0095] The CLIP text-image alignment model constitutes a feature extraction layer, and a ViT-B / 32 architecture is used to extract 512-dimensional visual and text features: v image = CLIP(I frame ), v text = CLIP(T slide ), and the deep semantic association is calculated by cosine similarity: The obtained SimCLIP is taken as the first cosine similarity between the video clip and the course content associated with the event type.

[0096] Step S307, obtaining the second cosine similarity between the audio text and the course content associated with the event type using the audio-text alignment model.

[0097] In some embodiments, the audio-text alignment model can be various models such as the BERT audio-text alignment model. The time sequence bias is eliminated by dynamic time warping of MFCC features, and the Whisper speech translation model and the BERT model are run in parallel with the CLIP text-image alignment model. The second cosine similarity between the audio text and the course content associated with the event type is obtained using the BERT model, and the second cosine similarity is simBERT = BERT(Whisper(A audio ), T summary ).

[0098] Step S308, in the case that the first cosine similarity is greater than the dynamic threshold value, and the corresponding second cosine similarity is greater than the audio-text threshold value, a standardized event tuple, a confidence score, and evidence trace information are generated.

[0099] The standardized event tuple includes user ID, event type, knowledge graph node, timestamp, and the like; the evidence trace information includes the path of the stored associated video frame and / or audio slice. A fault-tolerant mechanism can be used: 1. partial alignment processing, generating a partial_alignment event and marking the missing source when a single modality is valid; 2. drift compensation, calibrating the space-time reference every hour and dynamically adjusting the slice window; 3. configuring a confidence decay model. Real-time path can complete 5-second slice processing under GPU acceleration, and non-matching segments enter the night batch reprocessing queue.

[0100] The standardized event tuple event(user_id, event_type, knowledge_node, timestamp) is generated, that is, the standardized event tuple event(user ID, event type, knowledge graph node, timestamp) is generated, and the confidence score and the evidence trace information are attached. The evidence trace information includes the path of the stored associated video frame, audio slice, and the like. The confidence score can be a 0.0-1.0 score, and the confidence score can be determined using various methods, for example, the delay decay model can be used to determine the confidence score.

[0101] Figure 4 The flowchart for determining the confidence score of some embodiments of the resource calling method based on educational data according to the present disclosure is shown in FIG. 4 as follows: Figure 4

[0102] Step S401, at least one of the dynamic threshold value and the audio-text threshold value is adjusted according to the reference threshold value, the user type factor, the content type factor, and the delay compensation factor.

[0103] For example, at least one of the dynamic threshold value and the audio-text threshold value can be adjusted using the following formula:

[0104] T adjusted = T base × α user × β content × γ delay ;

[0105] Wherein: T adjusted is the adjusted dynamic threshold value or audio-text threshold value, T base is the reference threshold value of the dynamic threshold value or audio-text threshold value; α user is the user type factor (0.95-1.05); β content ​Content type factor (0.85-1.0); γ delay Delay compensation factor (0.98-1.0).

[0106] The reference threshold can be set to 0.85, etc. Dynamic threshold adjustment can be implemented for different scenarios, the threshold of high-value users can be relaxed to 0.80, and the threshold for complex chart recognition can be reduced to 0.75, which can balance recall rate and accuracy.

[0107] Step S402, determining the confidence score based on the basic confidence and the processing delay information.

[0108] The confidence score can be obtained by using various methods, and the confidence decay model can be set as:

[0109] conf eff = conf orig × e -0.5Δt ;

[0110] Where Δt is the processing delay information, conf orig is the basic confidence; the processing delay information can be the time information of the whole process from the start of receiving the education data to the generation of the standardized event tuple. For example, the confidence score is 0.92×e -0.5×Δt , wherein Δt is the processing delay information, and 0.92 is the basic confidence.

[0111] Context adaptive filtering can be achieved through a dynamic threshold adjustment algorithm, and the activation threshold (T user = 0.85×α×β×γ user ) is dynamically calculated based on the user type (α adj ) and the content type (α delay ) factor, so that the high-value user (α = 0.95) and the complex chart (β = 0.9) trigger threshold relaxation, and the decay compensation (γ = 0.98) is enabled when the processing delay is > 2 seconds. When simCLIP≥T adj and sim BERT ≥ 0.75, a complete alignment event is generated, and a degraded event with a missing_modalities label is output when there is a partial match; the influence of timing drift is processed through the final confidence decay model (conf eff = conf orig × e -0.5Δt ).

[0112] For spatio-temporal normalization processing and similarity analysis, etc., a processing speed of 5 ms / frame can be achieved on an NVIDIA A100 GPU, and in combination with a hierarchical pipeline design (real-time GPU acceleration + night batch reprocessing) and a double cache mechanism (course feature LRU cache + user session TTL cache), an alignment accuracy of 92.7% and a recall rate of 89.3% are achieved, providing high-precision spatio-temporal alignment event streams for downstream causal analysis.

[0113] Figure 5 A flowchart for generating a causal graph for some embodiments of the resource calling method based on educational data according to the present disclosure is shown in Figure 5 .

[0114] Step S501, generating a feature vector based on the event type in the user standardized event information, and constructing an event sequence matrix according to the feature vector.

[0115] Step S502, performing Granger test on each pair of event types in all event types using a vector autoregressive model according to the event sequence matrix, to determine the causal probability between each pair of event types.

[0116] Step S503, determining the occurrence probability between each pair of event types according to the causal probability between each pair of event types.

[0117] Step S504, generating a causal graph according to the event type and the occurrence probability between each pair of event types.

[0118] The nodes in the causal graph represent event types; the weighted edges between the nodes in the causal graph represent the occurrence probability between the nodes. The multi-modal event time series analysis method based on Granger causality test can uniformly encode heterogeneous events such as video pause, course annotation, and answer error into a time series matrix, and calculate cross-modal causal probability through a vector autoregressive model.

[0119] The occurrence probability between each pair of event types can be determined using a variety of methods according to the causal probability between each pair of event types. Figure 6 A flowchart for determining the occurrence probability for some embodiments of the resource calling method based on educational data according to the present disclosure is shown in Figure 6 .

[0120] Step S601, determining the event occurrence time corresponding to each pair of event types, and determining the time difference between the event occurrence time and the current time.

[0121] Step S602, determining a time decay factor according to the decay rate and the time difference.

[0122] Step S603, taking the product of the causal probability and the time decay factor as the occurrence probability between each pair of event types.

[0123] In some embodiments, Granger causality test can be used to analyze the event sequence (event sequence matrix), and a weighted causal graph (edge weight = causal probability x time decay factor) can be dynamically generated, and the graph can be updated every 100 events or 1 second window. The causal graph builder uses a dynamic incremental architecture, using Granger causality test as the core mathematical tool to analyze the causal relationship in the event sequence in real time.

[0124] Receive user standardized event information, the event type in the preprocessing layer is encoded into a 128-dimensional feature vector, and an event sequence matrix X is constructed t ∈R T×N Where T is the time step, and N is the event type dimension.

[0125] The Granger causality analysis layer performs vector autoregressive modeling:

[0126]

[0127] Where p = 3 is the maximum lag order.

[0128] Granger test is performed on each pair of event types (i, j):

[0129] (Event i does not cause event j);

[0130] Calculate the F statistic 1-p value is taken as the causal probability P ij .

[0131] The dynamic graph construction layer converts the causal probability into a weighted edge:

[0132]

[0133] Where w ij is the occurrence probability between each pair of event types, P ij is the causal probability between each pair of event types, and Δt is the time difference from the time of event occurrence to the current time, and λ is the decay rate, which can be 0.2, etc. For the event pair (event i, event j), Δt can take the time difference from the time of event i occurrence to the current time.

[0134] A causal graph corresponding to the user ID can be generated for each Kafka partition, such as Figure 7As shown, the nodes are "video pause", "knowledge point review", "courseware page turning", etc., the event type pair can be ("video pause", "knowledge point review"), etc., and the occurrence probability between the event type pair ("video pause", "knowledge point review") is w=0.91. The causal graphs of various user IDs can be combined to generate a global causal graph.

[0135] Figure 8 For some embodiments of the resource calling method based on education data according to the present disclosure, the flowchart for updating the causal graph is as shown in Figure 8

[0136] Step S801, after generating the causal graph, when it is determined that the causal probability between the new event type pair is greater than the causal rule threshold, the new event type pair is added to the causal graph.

[0137] Step S802, the new occurrence probability of the weighted edge in the causal graph is calculated, and the existing occurrence probability of the weighted edge in the causal graph is updated.

[0138] In some embodiments, the causal graph can be maintained through an incremental updating mechanism. When it is determined that the causal probability between two event types (a new event type pair) is greater than the causal rule threshold (for example, 0.5), that is, P ij >0.5, the nodes and the weighted edges corresponding to the two event types are added to the causal graph.

[0139] The existing edges in the causal graph are updated by an exponentially weighted moving average, as shown in the following formula:

[0140]

[0141] wherein, is the new occurrence probability of the weighted edge in the causal graph, and the existing occurrence probability of the weighted edge in the causal graph is updated. The node weight is dynamically adjusted by event counting (w i =log(1+count i )).

[0142] A triple optimization strategy can be set: 1. The sliding window mechanism retains the last 10 analysis windows, and an age attenuation weight 0.9 k is given; 2. A fast approximation algorithm is used to estimate the mutual information when resources are limited 3. A distributed computing architecture is used to achieve horizontal expansion through event stream partitioning, and each partition subgraph is synthesized into a global causal graph by a merger.

[0143] For example, the structure of the output causal graph contains dynamic nodes and weighted edges, and is updated every 1 second or 100 events:

[0144] ​

[0145] In a case where the occurrence probability of the weighted edge in the causal diagram is greater than the scheduling trigger threshold, it is determined to perform resource scheduling, and the scheduling trigger threshold can be set.

[0146] Figure 9 A flowchart for training a rule optimization model for some embodiments of the resource invocation method based on education data according to the present disclosure is shown in FIG. 8. Figure 8

[0147] In step S901, a reward function is constructed according to system resource state information.

[0148] The system resource state information includes delay rate, resource utilization rate, and false trigger rate, etc. Reinforcement learning is a kind of machine learning, which uses an agent to learn the optimal behavior strategy from trial and error through interaction with the environment to maximize the long-term cumulative reward.

[0149] The trigger threshold can be adjusted online based on reinforcement learning to realize the autonomous evolution of the rule optimization model. The rule optimization model can be a variety of models, and an intelligent decision-making framework based on reinforcement learning is adopted. The rule optimization model interacts with the environment in real time, and the reward function is the optimization target. The reward function is:

[0150] R = 0.6 x (1-delay rate) + 0.3 x resource utilization rate - 0.1 x false trigger rate.

[0151] In step S902, an adjustment action is determined by the rule optimization model according to the collected first system running state information.

[0152] The first system running state information includes delay rate, resource utilization rate, false trigger rate, etc. The adjustment action includes adjusting the scheduling trigger threshold, adjusting the proportion of resources that need to be scheduled, and a variety of scheduling rules, etc.

[0153] In step S903, after the adjustment action is executed, the reward value is calculated by the reward function according to the newly collected second system running state information. The second system running state information includes delay rate, resource utilization rate, false trigger rate, etc.

[0154] In step S904, a training sample is generated based on the first system running state information, the adjustment action, the reward value, and the second system running state information, and is put into an experience pool.

[0155] In step S905, when the model training condition is met, the training sample in the experience pool is used to optimize the reward value, and the parameter optimization of the rule optimization model is realized by policy gradient update to train the rule optimization model.

[0156] ​In some embodiments, the model training condition can be a plurality of conditions, for example, the number of samples in the experience pool is greater than a threshold, etc. The collected first system running state information s t is input into the rule optimization model, and the rule optimization model outputs an adjustment action a t, which includes adjusting the scheduling trigger threshold, adjusting the proportion of resources that need to be scheduled, etc.

[0157] After the adjustment action is executed, the reward value r t is calculated using the reward function according to the newly collected second system running state information s t+1, and based on s t, a t, r t, s t+1, a training sample is generated and stored in the experience replay pool. After 50 samples in the experience pool, a batch is randomly sampled, and the PPO algorithm is used to update the policy gradient, taking the reward value as the optimization objective, and the parameter optimization of the rule optimization model is realized through the policy gradient update, so as to train the rule optimization model. The trained rule optimization model is obtained.

[0158] The trained rule optimization model is used to adjust the scheduling trigger threshold and / or the proportion of resources that need to be scheduled, as well as the scheduling rules, etc. The resources that need to be scheduled include GPU resources, CDN resources, etc.

[0159] For the education scenario, a specific reward function can be set as R = 0.6 * (1 - delay rate) + 0.3 * resource utilization rate - 0.1 * false trigger rate, which is a composite formula containing various coefficient value ranges. The batch training method is triggered every 100,000 events to update the network, and the KL divergence constraint threshold (ε = 0.2) is included.

[0160] A variety of methods can be used to generate resource collaborative scheduling information using resource chain templates. Figure 10 The flowchart for generating resource collaborative scheduling information according to some embodiments of the resource invocation method based on education data of the present disclosure is shown in Figure 10

[0161] Step S1001, according to the causal diagram, the user standardized event information, and the proportion of resources that need to be scheduled, the trained analysis model is used to determine the event type that needs to be scheduled and the scheduling parameter. The analysis model can be a Transformer model, etc.

[0162] Step S1002, based on the scheduling parameter, the resource demand information is determined. The resource demand information includes computing resource demand information, storage resource demand information, network resource demand information, etc.

[0163] Step S1003, the resource chain template corresponding to the event type that needs to be scheduled is determined. The resource chain template includes a YAML template, etc. The YAML template is a reusable code structure defined using the YAML format, which is used to simplify configuration management and automated processes. ​

[0164] At step S1004, an instantiated template is generated as resource collaborative scheduling information according to the resource requirement information and the resource chain template.

[0165] In some embodiments, the collaborative topology of GPU cluster→video storage→edge CDN can be dynamically assembled by resolving the causal events and binding the resource chain template through the causal diagram, and millisecond-level resource mapping is supported.

[0166] The present disclosure can dynamically map causal events to multi-dimensional resource chains, implement collaborative scheduling of computing, storage, and network resources, and can adopt a dynamic binding architecture of a causal event resolution layer, a resource chain template matching layer, and a dynamic resource orchestration layer to achieve millisecond-level resource mapping. Through the collaborative work of event resolution, template matching, and dynamic orchestration, the key problems of high delay and low utilization caused by the splitting of existing static resource allocation and multi-modal data are solved.

[0167] The causal event resolution layer receives a causal diagram, user standardized event information, etc., classifies events through a pre-trained Transformer model, extracts key scheduling parameters, determines the type of events that need to be scheduled and scheduling parameters, and generates standardized resource requirement descriptions using dynamic calculation formulas, which can provide accurate input for subsequent resource binding. The event type can be "knowledge weakness", "live peak", "answering error", etc. The scheduling parameters can include the number of users, the size of associated knowledge point videos, the number of pre-cached copies, user geographic distribution entropy, dynamic weight, etc.

[0168] Various methods can be used to determine resource requirement information based on scheduling parameters. The resource requirement information includes computing resource requirement information, storage resource requirement information, and network resource requirement information. For example, the number of GPU instances = ceil(user number / 1000); video storage = associated knowledge point video size x pre-cached copy number; the number of CDN nodes = user geographic distribution entropy x dynamic weight.

[0169] The resource requirement information can be a standardized resource requirement description (JSON format):

[0170] {

[0171] "event_type":"knowledge_weakness",

[0172] "compute":{"gpu_instances":5,"inference_model":"matrix_solver"},

[0173] "storage": {"video_url": "cdn: / / matrix_tutorial.m3u8", "cache_replicas": 3},

[0174] "network": {"cdn_nodes": ["shanghai", "beijing"], "priority_ports":

[3478] }

[0175] }

[0176] The resource chain template matching layer is based on a predefined YAML template library and realizes dynamic instantiation of resource chains through a variable injection mechanism. The YAML template describes resource dependency relationships using a directed acyclic graph (DAG), supports conditional logic and dynamic parameter replacement, and enables the system to flexibly adapt to changes in the requirements of different educational scenarios. According to the event type, a predefined YAML template is matched, and parameters are dynamically filled to generate an executable scheduling scheme.

[0177] The template library stores resource dependency relationships for typical educational scenarios. For example, the YAML template for a knowledge weakness scenario is as follows:

[0178]

[0179]

[0180] Dynamic variable injection:

[0181] Replace runtime parameters (such as user_count=5000→gpu_instances=5) through the ${variable} syntax.

[0182] Support conditional logic (such as if user_count>10000 then replicas=5).

[0183] Output: instantiated resource chain description, such as:

[0184] {

[0185] "compute": {"gpu_instances": 5, "model": "matrix_solver"},

[0186] "storage": {"prefetch": ["cdn: / / matrix_tutorial.m3u8"], "replicas": 3},

[0187] "network": {"cdn_nodes": ["shanghai", "beijing"], "qos": "high"}

[0188] }

[0189] By selecting the "base template" corresponding to the scene (for example, the resource chain template of the "knowledge weakness" scene, which defines the dependency relationship of GPU, storage, and network) from the template library. According to the resource requirement information and the resource chain template, replace the variables in the template, instantiate the template, and generate the instantiated template as the resource collaborative scheduling information. Finally, verify the dependency relationship of the template, which can be verified using a directed acyclic graph (DAG), to ensure that the resource scheduling has a sequence and is not conflicted, for example, the storage prefetch does not perform in advance when the GPU is not ready.

[0190] The dynamic resource orchestration layer implements collaborative scheduling by parallelly calling underlying infrastructure APIs, including dependency-aware execution, fault-tolerant rollback mechanism, and probe-based state monitoring. Test data shows that this layer can reduce resource scheduling delay from 9.8 seconds in existing technologies to 380 milliseconds.

[0191] In some embodiments, a variety of methods can be used to perform system resource collaborative calling processing based on resource collaborative scheduling information. For example, according to the resource collaborative scheduling information, interfaces are called in parallel to perform resource scheduling processing resources to make resources work collaboratively; wherein the interfaces include K8S API for expanding GPU Pods, CDN preheating interface, interface for configuring SDN flow table, etc.

[0192] In terms of computing resources, Kubernetes API is called to expand GPU Pods in real time, where the number of instances is determined by dividing the number of event users by 1000 and rounding up. On the storage resource, the relevant content is pre-cached to the edge node through the CDN preheating interface, and the cached content is determined based on the knowledge node associated with the event. On the network resource, the SDN flow table is configured to prioritize network transmission of critical ports.

[0193] Load balancing can be performed, and link weights are dynamically allocated based on a resource topology graph, for example, when the current load in the Shanghai area exceeds 70%, 40% of the traffic is redirected to the Beijing area. Wait for the resources such as computing, storage, and network to be ready, and the timeout time is set to 200 ms, and finally generate a resource scheduling execution status report.

[0194] The present disclosure can describe the GPU→storage→CDN topology relationship through a YAML template, and dynamically inject runtime parameters; it can be based on a resource priority allocation strategy based on causal event weight, including the specific implementation of K8s expansion and contraction and SDN flow table linkage.

[0195] Based on resource coordination scheduling information, the system resource coordination calling process is started in parallel to start three types of tasks: expanding the GPU cluster with K8s, pre-storing videos with CDN, and configuring the network SDN to reduce the total time consumption. The resource dependency relationship is handled, for example, before the GPU cluster is ready, the status is continuously detected, and after the GPU is ready, the storage prefetch is triggered to avoid resource idling. The execution result is monitored: the GPU health status, CDN cache hit rate, network delay, and other data are collected to determine whether the scheduling is successful, providing a basis for subsequent optimization.

[0196] The present disclosure can upgrade the resource scheduling of the education scene from "manual configuration and static allocation" to "event-driven, dynamic adaptation, and self-optimization". For example, during the peak of live classes, automatically expand the GPU, pre-store the video, and schedule the CDN; after class, optimize the scheduling parameters next time according to the actual running data, which not only improves user experience (reduces latency) but also improves resource utilization (reduces idling), solving the traditional solution of "high latency, poor adaptation, and waste".

[0197] In some embodiments, thread-level decomposition tasks can be performed, and independent running is used for feature extraction tasks such as video, audio, and text to avoid serial bottlenecks; a non-blocking pipeline is formed through feature extraction, cross-modal alignment, and causal analysis to maximize hardware utilization; SIMD instructions are used to accelerate matrix operations and optimize feature vector calculations; independent threads are used for video, audio, and text processing, and zero-copy memory exchange is performed, shared memory pool management is used for feature data, and serialization / deserialization overhead is avoided; BPF (Berkeley Packet Filter) is used to monitor thread load in real time, and task allocation is dynamically adjusted.

[0198] Hardware acceleration integration can be used, NVIDIA NVENC hardware encoding + TensorRT can be used for 3D CNN for video; Intel AVX-512VNNI instructions can be used to optimize MFCC feature extraction for audio; FPGA can be used to accelerate BERT attention calculation for text.

[0199] Experiments show that the causal analysis delay under 10,000 concurrent requests can be compressed to ≤50ms, the efficiency is improved by 10 times, and 99.2% of 100,000 concurrent requests complete full-link processing within 48ms; efficient collaborative processing of multi-modal data such as video, audio, and text is achieved to ensure that end-to-end analysis is completed within ≤50ms of delay, which is suitable for 100,000 concurrent education scenarios.

[0200] The resource calling method based on education data provided by the present disclosure can improve the accuracy of causal identification of multi-modal education data, reduce the false triggering rate of resource calling, improve the accuracy of resource pushing, automatically trigger the cooperative scheduling of heterogeneous resources through causal events, improve the efficiency of dynamic resource cooperation, reduce the processing delay of multi-modal data, enhance the adaptability of rules, reduce the system response delay, effectively improve the utilization rate of cloud computing resources, reduce the platform operation cost, and can be applied to diversified application scenarios, and can significantly improve the service quality and operation efficiency of existing online education platforms.

[0201] Figure 11 A module schematic diagram of some embodiments of the resource calling device based on education data according to the present disclosure. As shown in the figure, the electronic device can include a memory 1101, a processor 1102, a communication interface 1103, and a bus 1104. The memory 1101 is used to store instructions, the processor 1102 is coupled to the memory 1101, and the processor 1102 is configured to execute the instructions stored in the memory 1101 to implement the resource calling method based on education data described above. Figure 11

[0202] The memory 1101 can be a high-speed RAM memory, a non-volatile memory, etc., and the memory 1101 can also be a memory array. The memory 1101 can also be divided into blocks, and the blocks can be combined into a virtual volume according to certain rules. The processor 1102 can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement the resource calling method based on education data of the present disclosure.

[0203] In some embodiments, the present disclosure provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the method in any of the above embodiments.

[0204] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of readable storage medium can include: electrical connection with one or more conductive wires, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.​

[0205] Embodiments of the present disclosure can also be a computer program product, which includes computer program instructions, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present disclosure described in the above “Exemplary Methods” section of the specification.

[0206] The resource calling method, device, computer readable storage medium and computer program product based on education data in the above embodiments can improve the accuracy of causal identification of multi-modal education data, reduce the false triggering rate of resource calling, improve the accuracy of resource pushing, improve the dynamic resource coordination efficiency; can reduce the processing delay of multi-modal data, enhance the rule adaptability, reduce the system response delay, effectively improve the utilization rate of cloud computing resources, reduce the platform operation cost, can be suitable for diversified application scenarios, can significantly improve the service quality and operation efficiency of the existing online education platform, and improve the user experience.

[0207] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be pointed out that the advantages, advantages, effects and the like mentioned in the present disclosure are only examples and not limitations, and these advantages, advantages, effects and the like cannot be considered as the must-have of each embodiment of the present disclosure. In addition, the above specific details are only for the purpose of example and for the purpose of understanding, and the above details do not limit the present disclosure to the above specific details.

[0208] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0209] The block diagrams of the devices, apparatuses, equipment, systems involved in the present disclosure are only exemplary examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any way. Words such as “include”, “contain”, “have” and the like are open words, which means “including but not limited to”, and can be used interchangeably. The words “or” and “and” used herein mean the word “and / or”, and can be used interchangeably unless the context clearly indicates otherwise. The word “such as” used herein means the phrase “such as but not limited to”, and can be used interchangeably.

[0210] It is also important to note that the construction and arrangement of the devices, equipment, and methods as shown in the various examples is illustrative only. Although only a few embodiments have been described, many modifications, changes, and substitutions can be made by one having ordinary skill in the art without departing from the application. Therefore, the above description is not intended to limit the scope of the application and the spirit, which is defined by the appended claims.

[0211] The above description of disclosed aspects is intended to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0212] The above description has been presented for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the disclosure to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those of ordinary skill in the art will appreciate a variety of modifications and alterations to the described embodiments which embody the principles of the disclosure and fall within the scope of the disclosure.

Claims

1. A method for resource retrieval based on educational data, comprising: The received educational data undergoes metadata extraction, spatiotemporal alignment processing, and quality detection to generate behavioral event information corresponding to the educational data. The educational data includes videos, audios, courseware content, and behavior logs; the behavior logs include event types and event times; and the behavior event information includes user ID and timestamp. Based on the user ID, the behavioral event information and the corresponding educational data are sent to the Kafka partition corresponding to the user ID, wherein the user ID and the Kafka partition correspond one-to-one. Based on the behavioral event information in the Kafka partition and the educational data, spatiotemporal normalization and similarity analysis are performed to generate standardized user event information. The user standardized event information includes standardized event tuples, confidence scores, and evidence tracing information. The standardized event tuple includes user ID, event type, knowledge node, and timestamp. Based on the user standardized event information, Granger causality analysis is performed to generate a causal graph; When resource scheduling is triggered based on the cause-effect graph, resource collaborative scheduling information is generated using the resource chain template based on the cause-effect graph and the user standardized event information. Based on the resource coordination scheduling information, system resource coordination invocation is performed, and the system resource status is monitored.

2. The method of claim 1, wherein, The quality detection process includes quality grading; the process of extracting metadata, performing spatiotemporal alignment, and conducting quality detection on the received educational data to generate behavioral event information corresponding to the educational data includes: Metadata is extracted from the educational data to obtain metadata corresponding to the educational data. The metadata includes the timestamp, coordinate information, user ID, and data feature information. The educational data is subjected to the spatiotemporal alignment process, which includes coordinate transformation, cross-device clock synchronization based on the PTP protocol, and resampling compensation. The educational data is processed based on QoS policies. Generate the behavior event information in JSON format, wherein the behavior event information further includes a global event ID, data type label, data information, and service level.

3. The method of claim 2, wherein, The step of generating standardized user event information by performing spatiotemporal normalization processing and similarity analysis based on the behavioral event information in the Kafka partition and the educational data includes: The video and audio are sliced ​​to generate video slices and audio slices; Based on the content of the courseware, determine the corresponding knowledge graph nodes in the knowledge graph; The video slices are spatially transformed using a spatiotemporal transformation matrix; The audio text corresponding to the audio slice is obtained using a speech recognition model; Based on the timestamps of the behavioral event information, determine the video clips, courseware content, and audio text associated with the event type; The first cosine similarity between video clips and courseware content associated with event types is obtained using an image-text alignment model. obtaining a second cosine similarity between the audio text associated with the event type and the course content by using an audio-text alignment model; generating the standardized event tuple, the confidence score, and the evidence trace information when the first cosine similarity is greater than a dynamic threshold and the corresponding second cosine similarity is greater than an audio-text threshold; wherein the standardized event tuple comprises a user ID, an event type, a knowledge graph node, and a timestamp; and the evidence trace information comprises a path of a stored associated video frame and / or audio slice.

4. The method of claim 3, comprising: adjusting at least one of the dynamic threshold and the audio-text threshold according to a baseline threshold, a user type factor, a content type factor, and a delay compensation factor; determining the confidence score based on a basic confidence and processing delay information.

5. The method of claim 3, wherein, the Granger causality analysis based on the user standardized event information comprises: generating a feature vector based on the event types in the user standardized event information, and constructing an event sequence matrix according to the feature vector; performing Granger test on each pair of event types in all event types by using a vector autoregressive model according to the event sequence matrix, to determine a causality probability between the each pair of event types; determining an occurrence probability between the each pair of event types according to the causality probability between the each pair of event types; generating the causality graph according to the event types and the occurrence probability between the each pair of event types; wherein a node in the causality graph represents an event type, and a weighted edge between the nodes in the causality graph represents an occurrence probability between the nodes.

6. The method of claim 5, wherein, the determination of the occurrence probability between the each pair of event types according to the causality probability between the each pair of event types comprises: determining an event occurrence time corresponding to the each pair of event types, and determining a time difference between the event occurrence time and a current time; determining a time decay factor according to a decay rate and the time difference; taking a product of the causality probability and the time decay factor as the occurrence probability between the each pair of event types.

7. The method of claim 5, further comprising: after generating the causality graph, adding a new pair of event types to the causality graph when a causality probability between the new pair of event types is greater than a causality rule threshold; and calculating a new occurrence probability of a weighted edge in the causality graph, and updating an existing occurrence probability of the weighted edge in the causality graph.

8. The method of claim 5, comprising: determining to perform resource scheduling when an occurrence probability of a weighted edge in the causality graph is greater than the dispatch trigger threshold.

9. The method of claim 8, comprising: constructing a reward function according to a delay rate, a resource utilization rate, and a false trigger rate; determining an adjustment action by using a rule optimization model according to collected first system running state information, wherein the first system running state information comprises the delay rate, the resource utilization rate, and the false trigger rate; and the adjustment action comprises adjusting the dispatch trigger threshold and adjusting a proportion of resources that need to be scheduled. After the adjustment action is performed, a reward value is calculated by using the reward function according to newly collected second system running state information, wherein the second system running state information includes a delay rate, a resource utilization rate, and a false triggering rate; Based on the first system running state information, the adjustment action, the reward value, and the second system running state information, a training sample is generated and put into an experience pool; When a model training condition is met, the training sample in the experience pool is used to optimize the reward value as an optimization target, and a parameter optimization of a rule optimization model is realized by a policy gradient update to train the rule optimization model.

10. The method of claim 9, comprising: adjusting the scheduling trigger threshold and / or adjusting the proportion of resources requiring scheduling by using the trained rule optimization model; wherein the resources requiring scheduling include GPU resources and CDN resources.

11. The method of claim 9, wherein, The generation of resource collaborative scheduling information by using the resource chain template based on the causal diagram and the user standardized event information comprises: determining an event type requiring resource scheduling and a scheduling parameter by using a trained analysis model based on the causal diagram, the user standardized event information, and the proportion of resources requiring scheduling; determining resource requirement information based on the scheduling parameter, wherein the resource requirement information includes computing resource requirement information, storage resource requirement information, and network resource requirement information; determining a resource chain template corresponding to the event type requiring resource scheduling, wherein the resource chain template includes a YAML template; generating the instantiated template as the resource collaborative scheduling information based on the resource requirement information and the resource chain template.

12. The method of any one of claims 1 to 11, wherein, The system resource collaborative calling processing based on the resource collaborative scheduling information comprises: performing resource scheduling processing resources in parallel by calling an interface based on the resource collaborative scheduling information, so that resources work collaboratively; wherein the interface includes a K8S API for expanding GPU Pods, a CDN preheating interface, and an interface for configuring an SDN flow table.

13. A resource retrieval device based on educational data, characterized in that, comprise: a memory; and a processor coupled to the memory, the processor being configured to execute a method as claimed in any one of claims 1 to 12 based on instructions stored in the memory.

14. A computer readable storage medium storing computer instructions, the instructions being executed by a processor to implement the steps of the method of any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1 to 12.

Images