A course scheduling optimization method and system based on micro-expression information

Abstract

This invention belongs to the field of big data mining technology and discloses a method and system for optimizing course scheduling based on micro-expression information. The method includes: acquiring video streams from a binocular camera and light intensity and color temperature data from environmental sensors; inputting the light intensity and color temperature data as bias terms into a spatiotemporal graph convolutional network, weighting and correcting the node features of the graph convolutional layer, and outputting a facial muscle movement unit feature sequence that eliminates environmental interference; inputting the curve into a multi-objective genetic algorithm scheduling engine, marking time segments with pressure peaks exceeding a preset threshold as non-scheduling penalty terms, and solving the problem using teacher resource idleness and classroom physical resource availability as boundary conditions to generate a globally optimal course scheduling timetable. This invention eliminates feature bias caused by changes in ambient light and couples the distribution of scheduling time with dynamic cognitive load, avoiding the stacking of consecutive high cognitive stress periods.

Description

Technical Field

[0001] This invention belongs to the field of big data mining technology and discloses a method and system for optimizing course scheduling based on micro-expression information. Background Technology

[0002] Existing video-based classroom monitoring and scheduling technologies involve installing cameras in classrooms to capture video footage. This footage is then fed into facial expression recognition models to extract students' facial features, which are used to calculate fatigue or interest levels for the entire lesson. At the scheduling level, the academic affairs scheduling system allocates time based on teachers' schedules and the availability of physical resources in classrooms, creating a fixed timetable. Some systems use fatigue indicators obtained from video analysis as post-class evaluation data to provide feedback to teachers or to adjust the difficulty of subsequent courses at a macro level. When processing video features, conventional facial expression recognition models directly extract features from the captured images, relying on the model's generalization ability to handle changes in ambient lighting.

[0003] Because conventional facial expression recognition models directly process raw images and only use the recognition results as a basis for post-class assessment or macro-level adjustments, the facial muscle movement unit features extracted by the model will be biased when the lighting intensity in the classroom changes dynamically. When these biased features are used to generate timetables, the scheduling system will receive incorrect cognitive load signals, making it impossible for the timetable algorithm to truly reflect changes in students' cognitive stress at the micro-temporal dimension. Consequently, when constructing timetables, the scheduling algorithm cannot couple and constrain the availability of physical resources with the students' actual dynamic cognitive load. Summary of the Invention

[0004] The purpose of this invention is to provide a solution that can effectively address the problems described in the background section.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The course scheduling optimization method based on micro-expression information includes: acquiring video streams captured by binocular cameras deployed in the classroom and light intensity and color temperature data collected by environmental sensors; The video stream and the light intensity and color temperature data are input into a spatiotemporal graph convolutional network. The light intensity and color temperature data are used as bias terms to perform weighted correction on the node features of the graph convolutional layer in the spatiotemporal graph convolutional network, and the facial muscle motion unit feature sequence that eliminates environmental interference is output. The facial muscle movement unit feature sequence is time-aligned with the course timestamp, the frequency of feature changes within a set time window is calculated, and a cognitive stress time series curve in the class dimension is generated. The cognitive stress time series curve is input into the multi-objective genetic algorithm scheduling engine. Time segments where the stress peak exceeds the preset threshold are marked as non-scheduling penalty items. The solution is performed with the teacher resource idle status and the classroom physical resource availability status as boundary conditions to generate a globally optimal course scheduling timetable.

[0006] Preferably, the node features of the graph convolutional layer in the spatiotemporal graph convolutional network are weighted and corrected using the light intensity and color temperature data as bias terms, including: The light intensity and color temperature data are converted into normalized environmental feature vectors; In each graph convolution operation of the spatiotemporal graph convolutional network, the normalized environment feature vector is multiplied bitwise with the output feature matrix of the current layer graph convolution kernel to generate an environment-corrected feature matrix. The environmental correction feature matrix is ​​input into the activation function to obtain the facial muscle movement unit feature sequence that eliminates environmental interference.

[0007] Preferably, the frequency of feature changes within a set time window is calculated to generate a cognitive stress time-series curve at the class level, including: The activation states of the frowning and blinking action units are extracted from the facial muscle movement unit feature sequence; Within the set time window, the activation frequency of the frowning action unit and the duration of continuous activation of the blinking action unit are counted. Substitute the activation frequency and the continuous activation duration into a preset pressure conversion function to obtain the pressure value corresponding to the set time window. By stitching together the stress values ​​corresponding to each set time window in chronological order, a cognitive stress time-series curve for the class dimension is generated.

[0008] Preferably, time segments where the pressure peak exceeds a preset threshold are marked as non-scheduling penalty items, and the solution is performed using the teacher resource idleness and classroom physical resource availability as boundary conditions, including: Construct an initial population, where each individual in the initial population represents a course time allocation scheme; Calculate the fitness value for each individual, which is a weighted sum of the unscheduling penalty, the teacher resource conflict penalty, and the classroom resource conflict penalty; When the fitness value meets the convergence condition or the number of iterations reaches the upper limit, the individual with the lowest fitness value is output as the globally optimal course schedule.

[0009] Preferably, acquiring video streams from binocular cameras deployed in the classroom and light intensity and color temperature data from environmental sensors includes: The binocular camera is used to acquire left-eye and right-eye views, and stereo matching of the left-eye and right-eye views is performed to generate three-dimensional point cloud data containing depth information. The three-dimensional point cloud data is used as the video stream. The environmental sensor and the binocular camera are synchronized using the Network Time Protocol (NTP) to assign a unified timestamp to the light intensity and color temperature data, as well as to each frame of the video stream.

[0010] Preferably, after generating the globally optimal course schedule, the process also includes: The globally optimal course schedule is compared with the currently executing initial course schedule to extract the target course segments that have undergone time changes. The teaching content identifier, original timestamp, and updated timestamp corresponding to the target course segment are encapsulated into a scheduling adjustment instruction; The scheduling adjustment instruction is sent to the teacher's terminal and the classroom display device for display.

[0011] Preferably, after generating the facial muscle movement unit feature sequence that eliminates environmental interference, the method further includes: Extract the facial region mask from the video stream; When the area of ​​the connected component of the facial region mask is less than a preset area threshold, it is determined that there is local occlusion; In the spatiotemporal graph convolutional network, determine the spatial neighbor nodes corresponding to the occluded nodes; Using the environmental correction feature matrix of the spatial neighborhood nodes, weights are calculated through a graph attention mechanism to aggregate and complete the features of the occluded nodes, resulting in the completed facial muscle movement unit feature sequence.

[0012] Preferably, after generating the cognitive stress time-series curve for the class dimension, the method further includes: Acquire data on the teacher's movement trajectory within the classroom; The spatial distance between the teacher and the student's face within the set time window is determined based on the movement trajectory data. When the spatial distance is less than the near-distance interaction threshold, the pressure value corresponding to the set time window is marked as the interaction interference value; The interaction interference value in the cognitive stress time series curve is replaced with the stress value of the adjacent time window using the median filtering algorithm to obtain the filtered cognitive stress time series curve.

[0013] Preferably, after calculating the fitness value of each individual, the method further includes: Calculate the first derivative of the cognitive stress time series curve in each set time window to obtain the stress change rate sequence; The windows in the pressure change rate sequence that are greater than the change rate threshold are identified as high steep increase windows; When performing crossover and mutation operations on the initial population, the mutation probability of the gene segment corresponding to the high steep growth window is increased, while the mutation probability of the gene segment corresponding to other windows is decreased, thereby generating a progeny population.

[0014] A course scheduling optimization system based on micro-expression information includes: a data acquisition device for acquiring video streams captured by binocular cameras deployed in the classroom and light intensity and color temperature data collected by environmental sensors; The feature extraction device is used to input the video stream and the light intensity and color temperature data into the spatiotemporal graph convolutional network, and use the light intensity and color temperature data as a bias term to perform weighted correction on the node features of the graph convolutional layer in the spatiotemporal graph convolutional network, and output a facial muscle motion unit feature sequence that eliminates environmental interference. The stress calculation device is used to time-align the facial muscle movement unit feature sequence with the course timestamp, calculate the frequency of feature changes within a set time window, and generate a cognitive stress time series curve in the class dimension. The scheduling engine device is used to input the cognitive stress time series curve into the multi-objective genetic algorithm scheduling engine, mark the time segment where the stress peak exceeds the preset threshold as a penalty item for not being able to schedule classes, and solve the problem with the teacher resource idle status and the classroom physical resource availability status as boundary conditions to generate a globally optimal course scheduling timetable.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention acquires light intensity and color temperature data from environmental sensors, inputs this data as bias terms into a spatiotemporal graph convolutional network, and performs weighted correction on the node features of the graph convolutional layer. Physical environment parameters are directly introduced during the feature extraction stage to correct image feature deviations caused by changes in ambient light, outputting a facial muscle movement unit feature sequence that eliminates environmental interference. The corrected feature sequence is aligned with course timestamps to generate a cognitive stress time series curve, which is then input into a multi-objective genetic algorithm scheduling engine. The scheduling engine marks time segments where stress peaks exceed a preset threshold as non-schedulable penalty terms, solving the problem with teacher resource availability and classroom physical resource availability as boundary conditions. This ensures that the generated course schedule satisfies physical resource constraints while directly coupling the scheduling time distribution with the dynamic cognitive load after eliminating environmental interference.

[0016] 2. This invention uses a binocular camera to collect 3D point cloud data containing depth information as a video stream. It utilizes a network time protocol to synchronize the environmental sensors and the binocular camera, assigning a unified timestamp to provide a physical benchmark for subsequent time alignment. When calculating the frequency of feature changes within a set time window, the activation states of frowning and blinking action units are extracted. The activation frequency and duration are statistically analyzed and substituted into a stress conversion function to convert specific facial muscle movements into cognitive stress values. During the scheduling engine's solution process, the weighted sum of unschedulable class penalties, teacher resource conflict penalties, and classroom resource conflict penalties is used as the fitness value. Iterative calculations are used to find the individual with the lowest fitness value, avoiding the stacking distribution of consecutive high cognitive stress periods within the same class's scheduling. Attached Figure Description

[0017] Figure 1 This is the main flowchart of the course scheduling optimization method of the present invention; Figure 2 This is a flowchart illustrating the environmental feature correction process of the present invention. Figure 3 This is a flowchart illustrating the generation of the cognitive stress time-series curve according to the present invention. Figure 4 This is a flowchart of the scheduling engine solution process of the present invention; Figure 5 This is a flowchart illustrating the partial occlusion feature completion process of the present invention. Figure 6 This is a flowchart of the interactive interference filtering process of the present invention. Detailed Implementation

[0018] Please refer to the attached document. Figure 1 This embodiment provides a course scheduling optimization method based on micro-expression information, implemented by a processing unit deployed on a local server in the smart classroom or a cloud-based academic management platform. The processing unit first acquires video streams from binocular cameras deployed in the classroom, as well as light intensity and color temperature data from environmental sensors. The processing unit establishes a stable communication link with the binocular cameras and environmental sensors via wired Ethernet or wireless LAN. The transmission latency of this communication link is lower than a preset latency threshold, ensuring the real-time nature of the acquired data.

[0019] The binocular camera continuously captures video images from the classroom at a fixed frame rate. Each frame includes both left and right viewpoints. The video stream is encoded and transmitted frame by frame using a lossless compression format to avoid loss of image details and subsequent feature extraction errors. The environmental sensor samples data at the same frequency as the binocular camera, outputting light intensity data in lux and color temperature data in Kelvin. Each set of environmental parameters corresponds to a specific sampling time.

[0020] Please refer to the attached document. Figure 2 The processing unit inputs the acquired video stream, along with illumination intensity and color temperature data, into a spatiotemporal graph convolutional network. Using the illumination intensity and color temperature data as bias terms, it weights and corrects the node features of the graph convolutional layers in the spatiotemporal graph convolutional network, outputting a facial muscle motion unit feature sequence that eliminates environmental interference. The processing unit first converts the collected illumination intensity and color temperature data into a normalized environmental feature vector. The normalization process is implemented using the following formula: .

[0021] in, This is a normalized environmental feature vector with a dimension of 2×1; This is the measured value of the light intensity at the current acquisition time; This is the lower limit of the light intensity measurement range for the environmental sensor; This represents the upper limit of the light intensity measurement range for the environmental sensor. This is the measured color temperature value at the current acquisition time; This is the lower limit of the color temperature measurement range for the environmental sensor; This represents the upper limit of the color temperature measurement range of the environmental sensor. Each element of the normalized feature vector takes values ​​in the range [0,1], eliminating the influence of different physical dimensions on the subsequent weighted correction process.

[0022] Spatiotemporal graph convolutional networks are composed of It consists of alternating layers of spatial graph convolutional modules and temporal convolutional modules. The input is a positive integer greater than or equal to 3. The input to the spatial graph convolution module is the graph structure data corresponding to facial muscle motion units. The node set of this graph structure is a preset set of facial muscle motion units, where each node corresponds to an independent muscle motion unit in the facial motion coding system. The edge set of the graph structure represents the spatial adjacency relationships between nodes, which are pre-defined based on the physiological connections of facial muscles. Each spatial graph convolution module contains at least one graph convolutional layer, and the input to the graph convolutional layer is the node feature matrix of the current layer. Where N is the number of nodes. The feature dimension for each node. The core computation process of the graph convolutional layer is implemented through the following formula: .

[0023] in, The output feature matrix of the graph convolutional layer; This is the normalized adjacency matrix; Let be the learnable weight matrix of the graph convolutional layer.

[0024] The processing unit uses the aforementioned normalized environment feature vector as a bias term and performs a positional multiplication with the output feature matrix of the current layer's convolution kernel to generate an environment-corrected feature matrix. The specific calculation process is implemented through the following formula: .

[0025] in, Correct the feature matrix for the environment; This is a bitwise multiplication operation; The environment bias matrix is ​​derived from the normalized environment eigenvectors. Obtained through mapping via a fully connected layer, the dimension and Completely identical. The activation function of the fully connected layer uses... The function ensures that the value range of each element in the environment bias matrix is ​​[0,1], so that the weighted correction process is a scaling adjustment of the original features, avoiding feature value overflow.

[0026] The processing unit inputs the environment-corrected feature matrix into the activation function to obtain the output features of the current graph convolutional layer. The activation function is a linear rectified activation function, the specific expression of which is: .

[0027] in, It is a learnable bias vector; The activation function is a linear rectifier, and its specific expression is: .

[0028] The temporal convolution module of the spatiotemporal graph convolutional network extracts temporal features from the feature sequence output by the spatial graph convolution module, capturing the dynamic changes of facial muscle motor units between consecutive frames. The temporal convolution module employs a one-dimensional causal convolution structure, with the receptive field of the convolution kernel covering multiple consecutive frames, ensuring that the extracted features reflect the continuous changes in micro-expressions. The final output of the spatiotemporal graph convolutional network is a facial muscle motor unit feature sequence free from environmental interference. Each element in the feature sequence corresponds to a facial muscle motor unit feature vector for one frame in the video stream. The dimension of the feature vector is consistent with the number of nodes, and the value of each dimension corresponds to the activation intensity of a facial muscle motor unit.

[0029] Please refer to the attached document. Figure 3The processing unit aligns the obtained facial muscle movement unit feature sequence with the course timestamps, calculates the frequency of feature changes within a set time window, and generates a cognitive stress time-series curve at the class level. The course timestamps come from the course schedule data in the academic management system and include the start time, end time, break time, and segment division time within the course. The processing unit aligns the acquisition time corresponding to each element of the facial muscle movement unit feature sequence with the course timestamps on the timeline, mapping the feature sequence onto the course timeline so that each feature element corresponds to a specific position on the course timeline. Linear interpolation is used during the mapping process to compensate for the deviation between the acquisition time and the course timestamp, ensuring complete synchronization between the feature sequence timeline and the course timeline.

[0030] The processing unit sets a sliding time window. Within each set time window, it counts the frequency of feature changes in the facial muscle motor unit feature sequence within the corresponding time period. The frequency of feature changes is calculated as follows: within the time window, when the activation intensity of a facial muscle motor unit exceeds a preset activation threshold, the motor unit is considered to have been effectively activated. The number of all effective activations within the time window is counted, which is the frequency of feature changes corresponding to that time window. The activation threshold is set based on the average activation intensity of the motor units under no cognitive stress.

[0031] The processing unit calculates the pressure value corresponding to each time window based on the frequency of characteristic changes. The pressure value calculation process is implemented through a preset pressure conversion function, the specific expression of which is: .

[0032] in, For the first The pressure values ​​corresponding to each time window; For the first The frequency of feature changes corresponding to each time window; A pre-defined stress transformation function is used, which is a monotonically increasing function to ensure that an increase in the frequency of feature changes corresponds to a synchronous increase in stress values. The processing unit concatenates the stress values ​​corresponding to each set time window in chronological order to generate a cognitive stress time-series curve at the class level. The horizontal axis of the curve represents the course time axis, and the vertical axis represents the stress value corresponding to each time window.

[0033] In this embodiment, the time window division rules for different stages of the course are shown in the table below: This table is used to standardize the time window division rules for different stages of the course, ensuring that a smaller time window length is used in the core lecture stage and classroom interaction stage where the course content density is high, thereby improving the temporal resolution of the cognitive stress time series curve, while a larger time window length is used in the course introduction stage and the break stage to reduce the consumption of computing resources.

[0034] Please refer to the attached document. Figure 4 The processing unit inputs the generated cognitive stress time series curve into the multi-objective genetic algorithm scheduling engine, marking time segments where the stress peak exceeds a preset threshold as non-scheduling penalty items. The engine solves the problem using teacher resource availability and classroom physical resource availability as boundary conditions to generate a globally optimal course schedule. The inputs to the multi-objective genetic algorithm scheduling engine include: the cognitive stress time series curve at the class level, teacher resource availability data, classroom physical resource availability data, and a preset scheduling period.

[0035] Teacher resource availability data includes the available and busy time slots for all instructors within the scheduling period. Busy time slots are those when instructors have scheduled classes, meetings, or other work, while available time slots are those when instructors can use the time slots for teaching. Classroom physical resource availability data includes the available and occupied time slots for all available classrooms within the scheduling period. Occupied time slots are those when classrooms have scheduled courses or activities, while available time slots are those when classrooms can be used for teaching.

[0036] The scheduling engine first processes the cognitive stress time-series curve, traversing all time segments within the scheduling period. It compares the stress peak value for each time segment with a preset stress threshold. When the stress peak value for a time segment exceeds the preset threshold, that time segment is marked as unschedulable and a penalty is applied. The scheduling engine constructs a solution space for scheduling optimization using teacher availability and classroom availability as boundary conditions. Specifically, all course teaching times must be scheduled within the available time slots of the corresponding teachers and classrooms, and teacher or classroom resource conflicts are not allowed.

[0037] The scheduling engine employs a multi-objective genetic algorithm. First, an initial population is constructed, where each individual represents a course time allocation scheme. Each individual's gene sequence corresponds to all time segments within the scheduling period, and each gene locus in the sequence corresponds to the teaching time arrangement of a course. The processing unit calculates the fitness value of each individual. The fitness value is a weighted sum of penalties for unschedulable courses, teacher resource conflicts, and classroom resource conflicts. The specific calculation process is implemented using the following formula: .

[0038] in, This represents the fitness value of an individual. This is the cumulative value of the penalty for courses that cannot be scheduled. This represents the cumulative value of the penalty for teacher resource conflicts. This is the cumulative value of the classroom resource conflict penalty item; , , These are the weight coefficients for the three penalty terms, and all weight coefficients are real numbers greater than 0.

[0039] The scheduling engine performs selection, crossover, and mutation operations on the initial population to generate offspring. It then iterates repeatedly until the fitness value meets the convergence condition or the number of iterations reaches a preset maximum. At this point, it stops iterating and outputs the course time allocation scheme corresponding to the individual with the lowest fitness value, which serves as the globally optimal course schedule. The convergence condition is that the change in the optimal fitness value of the population over a preset number of generations is less than a preset convergence threshold. When the convergence condition is met, the algorithm is considered to have converged to the globally optimal solution.

[0040] In this embodiment, by introducing light intensity and color temperature data collected by environmental sensors, the node features of the spatiotemporal graph convolutional network are weighted and corrected during the feature extraction stage to eliminate the interference of ambient light changes on feature extraction and obtain accurate facial muscle movement unit feature sequences. Based on the feature sequences, a cognitive stress time series curve at the class level is generated. The cognitive stress time series curve is used as the core basis for scheduling optimization. The scheduling engine of multi-objective genetic algorithm is used to solve the problem under the constraints of teacher resources and classroom resources. The generated course schedule can be coupled with the dynamic cognitive load of students to avoid course stacking during periods of high cognitive stress.

[0041] In one optional embodiment, when the processing unit inputs the video stream and illumination intensity and color temperature data into the spatiotemporal graph convolutional network, it further refines the process by using the illumination intensity and color temperature data as bias terms to perform weighted correction on the node features of the graph convolutional layer in the spatiotemporal graph convolutional network. Specifically, after the processing unit converts the collected illumination intensity and color temperature data into a normalized environment feature vector, it inputs the normalized environment feature vector into two fully connected layers, mapping the 2D environment feature vector into an environment bias matrix that matches the dimension of the output feature matrix of the graph convolutional layer. The activation function of the fully connected layer is the Sigmoid function, ensuring that the value range of each element of the environment bias matrix is ​​[0,1], so that the weighted correction process is a scaling adjustment of the original features, avoiding feature value overflow.

[0042] In each graph convolution operation of the spatiotemporal graph convolutional network, the processing unit multiplies the environment bias matrix obtained by mapping the normalized environment feature vector with the output feature matrix of the current layer's graph convolution kernel, generating an environment-corrected feature matrix. Each spatial graph convolution module in the spatiotemporal graph convolutional network contains two graph convolutional layers. The output feature matrix of each graph convolutional layer is multiplied with the corresponding environment bias matrix. The environment bias matrix is ​​updated in real time based on the light intensity and color temperature data at the current acquisition time, ensuring that the feature extraction process of each layer can be dynamically corrected according to the current environmental parameters. For the temporal convolutional layer in the spatiotemporal graph convolutional network, the processing unit also multiplies the environment bias matrix with the output feature matrix of the temporal convolutional layer, synchronously introducing the correction of environmental parameters during the temporal dimension feature extraction process, further eliminating the interference of ambient light changes on the feature sequence in the temporal dimension.

[0043] The processing unit inputs the environmental correction feature matrix into the activation function to obtain the facial muscle motion unit feature sequence that eliminates environmental interference. The activation function uses a leaky linear rectified activation function to avoid neuron death; the expression for the activation function is: .

[0044] in, The preset leakage coefficient has a value range of (0,1), preferably... The value is 0.01.

[0045] Please refer to the attached document. Figure 5 After generating facial muscle motion unit feature sequences to eliminate environmental interference, the processing unit performs occlusion detection and feature completion processing on the facial regions in the video stream. The processing unit first extracts facial region masks from the video stream. These masks are generated using a pre-trained facial segmentation model. The input to the facial segmentation model is a single frame image from the video stream, and the output is a binary mask image of the same size as the input image. Regions with a pixel value of 1 in the mask image correspond to detected facial regions, while regions with a pixel value of 0 correspond to non-facial regions. The processing unit performs connected component analysis on the generated facial region masks, calculating the area of ​​each connected component. The area of ​​a connected component is obtained by counting the number of pixels with a pixel value of 1 within the corresponding connected component in the mask image.

[0046] When the area of ​​the connected components of the facial region mask is less than a preset area threshold, the processing unit determines that the facial region has partial occlusion. The preset area threshold is set based on the average area of ​​the connected components of the facial region in the unoccluded state. Preferably, the preset area threshold is 60% of the average area of ​​the connected components of the facial region in the unoccluded state. When partial occlusion is determined, the processing unit determines the spatial neighbor nodes corresponding to the occluded node in the graph structure corresponding to the spatiotemporal graph convolutional network. The occluded node is the facial muscle motion unit node corresponding to the occluded facial region, and the spatial neighbor nodes are the adjacent nodes in the graph structure that are connected to the occluded node by an edge. The edge connection relationship is preset according to the physiological connection relationship of facial muscles to ensure that there is a physiological linkage between the facial muscle region corresponding to the spatial neighbor node and the facial muscle region corresponding to the occluded node.

[0047] The processing unit utilizes the environment-corrected feature matrix of spatial neighbor nodes, calculates weights through a graph attention mechanism, and aggregates and completes the features of occluded nodes to obtain the completed facial muscle motion unit feature sequence. The weight calculation process of the graph attention mechanism is implemented through the following formula: .

[0048] in, The attention weights are those of the neighboring nodes j corresponding to the occluded node i. For the occluded node The initial feature vector; Neighboring nodes Environmentally modified feature vectors; This is a feature vector concatenation operation; This is a learnable attention weight vector; Let i be the set of all spatial neighboring nodes of the occluded node i.

[0049] The completed feature vector of the occluded node is calculated using the following formula: .

[0050] in, The complete feature vector of the occluded node i; The Sigmoid activation function ensures that the numerical range of the padded feature vector remains consistent with that of the original feature vector.

[0051] In this embodiment, the mapping relationship between facial muscle motion unit nodes and their spatial neighborhoods is shown in the table below: This table is used to clarify the spatial neighborhood mapping relationship between facial muscle motion unit nodes, providing a basis for selecting neighborhood nodes for feature completion under partial occlusion. The selection of neighborhood nodes is based on the physiological connectivity of facial muscles, ensuring that the features of neighborhood nodes can accurately reflect the muscle motion state of the occluded nodes and improve the accuracy of feature completion.

[0052] In this embodiment, by normalizing the light intensity and color temperature data and mapping them to an environment bias matrix, feature weighting correction is performed in each layer of graph convolution and temporal convolution operations of the spatiotemporal graph convolutional network. Local occlusion detection is performed through connected component analysis of facial region masks. Based on the graph attention mechanism, the features of spatial neighborhood nodes are used to aggregate and complete the features of occluded nodes, which solves the feature extraction deviation problem caused by local occlusion and further improves the accuracy of facial muscle motion unit feature sequences.

[0053] In another optional embodiment, the processing unit aligns the facial muscle movement unit feature sequence with the course timestamps, calculates the frequency of feature changes within a set time window, and generates a cognitive stress time-series curve at the class level for further refinement. Specifically, the processing unit first aligns the facial muscle movement unit feature sequence with the course timestamps. The course timestamps come from the course schedule data of the academic management system and include the start time, end time, break time, and segment division time of each day within the scheduling period. Each time corresponds to a unified network time protocol clock reference. The processing unit maps and aligns the acquisition time corresponding to each feature element in the facial muscle movement unit feature sequence with the time axis of the course timestamps. During the mapping process, linear interpolation is used to compensate for the deviation between the acquisition time and the course timestamp time, ensuring that the time axis of the feature sequence is completely synchronized with the course time axis.

[0054] The processing unit extracts the activation states of the frowning and blinking action units from the facial muscle motor unit feature sequence. The frowning action unit corresponds to the AU4 unit in the facial motion coding system, and the blinking action unit corresponds to the AU45 unit in the same system. The activation states of these two action units are directly physiologically related to the cognitive stress level of the human body. The processing unit extracts the activation intensity of the frowning and blinking action units at each time point in the feature sequence. When the activation intensity exceeds a preset activation threshold, the action unit is determined to be active at that time point; when the activation intensity is lower than or equal to the preset threshold, the action unit is determined to be inactive at that time point. The activation threshold is set based on the average activation intensity of the action units under no cognitive stress conditions. Preferably, the activation threshold is three times the average activation intensity of the action units under no cognitive stress conditions.

[0055] The processing unit counts the activation frequency of the frowning action unit and the duration of continuous activation of the blinking action unit within a set time window. The activation frequency is counted as follows: within the set time window, the number of times the frowning action unit switches from an inactive state to an active state is counted. Each state switch is recorded as a valid activation, and the total number of valid activations is the activation frequency of the frowning action unit within that time window. The duration of continuous activation is counted as follows: within the set time window, the number of consecutive frames in which the blinking action unit is active is counted. The number of consecutive frames is multiplied by the time interval of a single frame to obtain the duration of that activation. The durations of all activation processes within the time window are summed to obtain the continuous activation duration of the blinking action unit within that time window.

[0056] The processing unit substitutes the activation frequency and duration into a preset pressure conversion function to obtain the pressure value corresponding to the set time window. The specific expression of the pressure conversion function is as follows: .

[0057] in, For the first The pressure values ​​corresponding to each time window; For the first The activation frequency of the frowning action unit within a time window; For the first The duration of continuous activation of the blinking action unit within a time window; , These are the weighting coefficients corresponding to activation frequency and duration of continuous activation, respectively. All weighting coefficients are real numbers greater than 0, and their values ​​are obtained by fitting pre-collected cognitive stress calibration data.

[0058] The processing unit concatenates the stress values ​​corresponding to each set time window in chronological order to generate a cognitive stress time-series curve at the class level. The generation process for the class-level cognitive stress time-series curve is as follows: the facial muscle movement unit feature sequences of all students in the class are processed separately to obtain the individual-user cognitive stress time-series curve for each student; the stress values ​​of all individual-user cognitive stress time-series curves within the same time window are averaged to obtain the class-level stress value; and the stress values ​​of all classes are concatenated in chronological order to generate the final class-level cognitive stress time-series curve.

[0059] In this embodiment, the correspondence between the activation state range of the action unit under different cognitive stress levels and the parameters of the stress conversion function is shown in the following table: This table is used to standardize the corresponding range of activation states of motor units and the weighting coefficients of the stress conversion function under different levels of cognitive stress. The weighting coefficients are dynamically adjusted according to different activation state ranges, so that the stress conversion function can more accurately reflect the contribution of frowning and blinking actions to cognitive stress under different stress levels, thereby improving the accuracy of stress value calculation.

[0060] Please refer to the attached document. Figure 6 After generating the cognitive stress time-series curve at the class level, the processing unit performs interactive interference filtering on the cognitive stress time-series curve. The processing unit first acquires the teacher's movement trajectory data within the classroom. This data is calculated from the video stream captured by binocular cameras deployed in the classroom. The processing unit then identifies the teacher target in the video stream using a pre-trained target detection model. Based on the 3D point cloud data acquired by the binocular cameras, it calculates the 3D spatial coordinates of the teacher target within the classroom and stitches these coordinates together in chronological order to generate the teacher's movement trajectory data within the classroom.

[0061] The processing unit determines the spatial distance between the teacher and students' faces within a set time window based on the movement trajectory data. The processing unit calculates the Euclidean distance between the teacher's 3D spatial coordinates and the 3D spatial coordinates of all students' faces within the set time window, and uses the average of all Euclidean distances as the spatial distance between the teacher and students' faces within that set time window. The students' 3D spatial coordinates are calculated from 3D point cloud data acquired by a binocular camera, and are synchronized with the acquisition time of the facial muscle movement unit feature sequence.

[0062] When the spatial distance is less than the close-range interaction threshold, the processing unit marks the pressure value corresponding to the set time window as an interaction interference value. The close-range interaction threshold is set according to the space size in the classroom; preferably, it is 1.5 meters. When the spatial distance between the teacher's and student's faces is less than 1.5 meters, it is determined that there is close-range classroom interaction between the teacher and student. This interaction process will cause changes in the student's facial muscle movements that are unrelated to cognitive pressure, thus causing abnormal fluctuations in the pressure value. Therefore, the pressure value corresponding to this time window is marked as an interaction interference value.

[0063] The processing unit uses a median filtering algorithm to replace the interaction interference values ​​in the cognitive stress time series curve with stress values ​​from adjacent time windows, resulting in a filtered cognitive stress time series curve. The specific implementation process of the median filtering algorithm is as follows: Centering on the time window marked as the interaction interference value, select the stress values ​​corresponding to a preset number of adjacent time windows. Sort the selected stress values, take the median of the sorted values ​​as the replacement value for the interaction interference value, and fill the corresponding position in the cognitive stress time series curve with the replacement value to complete the filtering process. The replacement process of median filtering is implemented through the following formula: .

[0064] in, The pressure value after replacement; This is for the median value operation; is the half-width of the filter window, and is a positive integer greater than or equal to 1; to The pressure values ​​corresponding to the adjacent time windows preceding the time window marked as interactive disturbance values; to The pressure value is the value of the adjacent time window following the time window marked as the interactive disturbance value.

[0065] In this embodiment, the activation states of the frowning and blinking action units, which are directly related to cognitive stress, are extracted, and the activation frequency and duration are statistically analyzed and substituted into the stress conversion function. The spatial distance between the teacher and the student's face is calculated using the teacher's movement trajectory data, and the stress value interference caused by close interaction is identified. The median filtering algorithm is used to replace the interaction interference value, thereby eliminating the interference of classroom interaction on the cognitive stress time series curve and further improving the accuracy of the cognitive stress time series curve.

[0066] In another alternative embodiment, the processing unit inputs the cognitive stress time series curve into the multi-objective genetic algorithm scheduling engine, marks the time segments where the stress peak exceeds a preset threshold as unschedulable penalty items, and performs further refinement processing by solving the process with the teacher resource idle status and the classroom physical resource availability status as boundary conditions to generate the globally optimal course schedule.

[0067] The processing unit first preprocesses the input cognitive stress time-series curve, dividing it into multiple consecutive time segments according to the scheduling cycle's time division rules. The length of each time segment is consistent with the teaching duration of a single course, preferably 45 minutes. The processing unit calculates the cognitive stress peak value within each time segment, which is the maximum stress value among all time windows within that segment. The peak value is compared with a preset stress threshold. When the peak value exceeds the threshold, the time segment is marked as an unschedulable time segment, and a non-schedulable penalty is set for it. The penalty value is a preset fixed penalty value, and the corresponding penalty value is accumulated when a course is scheduled in an unschedulable time segment.

[0068] The processing unit constructs the initial population for the multi-objective genetic algorithm. Each individual in the initial population represents a course time allocation scheme. The gene sequence of each individual uses integer encoding, and the length of the gene sequence is equal to the total number of courses to be scheduled. Each gene position in the gene sequence corresponds to a course to be scheduled, and the value of the gene position corresponds to the teaching time and classroom code of the course. The initial population generation process combines random generation with constraint filtering. First, a preset number of individuals are randomly generated. Constraints are checked on each individual, and individuals that do not meet the boundary conditions of teacher resource idleness and classroom physical resource availability are removed. New random individuals are added until the number of individuals in the initial population reaches the preset population size.

[0069] The processing unit calculates the fitness value for each individual. The fitness value is a weighted sum of the unschedulable course penalty, the teacher resource conflict penalty, and the classroom resource conflict penalty. The cumulative value of the unschedulable course penalty is the sum of the penalty values ​​corresponding to courses scheduled in unschedulable time slots within the individual's corresponding course time allocation scheme. The cumulative value of the teacher resource conflict penalty is the sum of the penalty values ​​corresponding to the number of times the same teacher is scheduled for multiple courses within the same time slot within the individual's corresponding course time allocation scheme. The cumulative value of the classroom resource conflict penalty is the sum of the penalty values ​​corresponding to the number of times the same classroom is scheduled for multiple courses within the same time slot within the individual's corresponding course time allocation scheme. The fitness value is calculated using the aforementioned weighted sum formula. The weight coefficients are set according to the priority of scheduling optimization. Preferably, the weight coefficient of the unschedulable course penalty is the highest, followed by the weight coefficients of the teacher resource conflict penalty and the classroom resource conflict penalty.

[0070] In this embodiment, the weight coefficient configuration rules for different penalty items during the scheduling optimization process are shown in the following table: This table is used to standardize the weight coefficient configuration of different penalty items in the scheduling optimization process. The value of the weight coefficient is adjusted according to the different needs of the scheduling scenario to ensure that the generated course schedule can meet the scheduling needs of different scenarios.

[0071] The processing unit performs a selection operation on the initial population using a tournament selection method. A predetermined number of individuals are randomly selected from the current population, and the individual with the lowest fitness value is selected to enter the offspring population. This process is repeated until the number of individuals in the offspring population reaches the predetermined population size. After the selection operation, the processing unit performs a crossover operation on the offspring population using a two-point crossover method. Two parent individuals are randomly selected, and two crossover points are randomly chosen. The gene segments of the two parent individuals between the two crossover points are exchanged, generating two new offspring individuals. The crossover operation has a predetermined fixed probability.

[0072] After calculating the fitness value of each individual, the processing unit adaptively adjusts the crossover and mutation operations of the genetic algorithm. Specifically, the processing unit calculates the first derivative of the cognitive stress time-series curve for each set time window to obtain the stress change rate sequence. The stress change rate is calculated using the following formula: .

[0073] in, For the first The rate of pressure change corresponding to each time window; For the first The pressure values ​​corresponding to each time window; For the first The pressure values ​​corresponding to each time window; This represents the time interval between two adjacent time windows.

[0074] The processing unit identifies windows in the pressure change rate sequence that exceed a certain threshold as steeply increasing windows. The threshold is set based on the mean and standard deviation of the pressure change rate sequence; preferably, it is the mean plus twice the standard deviation. The time segments corresponding to steeply increasing windows represent periods of rapid increase in students' cognitive stress. Scheduling during these periods significantly impacts students' cognitive load, thus requiring focused optimization in the mutation operation of the genetic algorithm.

[0075] When performing crossover and mutation operations on the initial population, the processing unit increases the mutation probability of gene segments corresponding to high-increase windows and decreases the mutation probability of gene segments corresponding to other windows, generating the offspring population. The adaptive mutation probability is calculated using the following formula: .

[0076] in, For the first The mutation probability of a gene segment corresponds to a time window; This is the set of time segments corresponding to all steeply increasing windows; For high mutation probability, the value ranges from 0.1 to 0.3; For low mutation probability, the value ranges from 0.01 to 0.05.

[0077] By adaptively adjusting the mutation probability, the genetic algorithm performs more mutation operations on gene segments corresponding to the steeply increasing window of cognitive pressure during the iteration process, thereby accelerating the optimization speed in that region. It also performs fewer mutation operations on gene segments corresponding to other windows, maintaining the stability of the optimized scheduling scheme and improving the convergence speed and solution accuracy of the algorithm.

[0078] The processing unit iteratively performs selection, crossover, and mutation operations, calculating the fitness value of individuals in each generation of the population. Iteration stops when the fitness value meets the convergence condition or the number of iterations reaches a preset maximum. The convergence condition is that the change in the optimal fitness value of the population over a preset number of generations is less than a preset convergence threshold. When the convergence condition is met, the algorithm is considered to have converged to the global optimum. After iteration stops, the processing unit outputs the course time allocation scheme corresponding to the individual with the lowest fitness value in the current population, which serves as the globally optimal course schedule.

[0079] After generating the globally optimal course schedule, the processing unit executes the generation and issuance of schedule adjustment instructions. The processing unit compares the generated globally optimal course schedule with the currently executing initial course schedule, comparing the teaching time and classroom arrangements for each course, and extracting target course segments with changed times. Target course segments are those courses whose teaching times have changed, including changes to the start time, end time, or teaching date.

[0080] The processing unit encapsulates the teaching content identifier, original timestamp, and updated timestamp corresponding to the target course segment into a scheduling adjustment instruction. The teaching content identifier is a unique identification code for the course, used to uniquely identify the teaching content of that course; the original timestamp is the teaching date, start time, and end time of the course in the initial course schedule; the updated timestamp is the teaching date, start time, and end time of the course in the globally optimal course schedule. The scheduling adjustment instruction is encapsulated using a preset communication protocol format to ensure that the receiving terminal can correctly parse the instruction content.

[0081] The processing unit sends the scheduling adjustment command to the teacher's terminal and classroom display devices via local area network (LAN) or wide area network (WAN). The teacher's terminal is a smartphone, tablet, or computer used by the instructor. Upon receiving the scheduling adjustment command, the teacher's terminal displays the adjusted course information on its interface, reminding the instructor of the change in course times. The classroom display devices are smart blackboards, electronic class signs, or other display devices deployed in the classroom. Upon receiving the scheduling adjustment command, the classroom display devices update the displayed course schedule information, ensuring that students in the classroom are promptly informed of the course schedule changes.

[0082] In this embodiment, the processing unit further refines the process of acquiring video streams from binocular cameras deployed in the classroom and light intensity and color temperature data from environmental sensors. Specifically, the processing unit uses the binocular cameras to acquire left-eye and right-eye views, performs stereo matching on the left-eye and right-eye views to generate 3D point cloud data containing depth information, and uses the 3D point cloud data as the video stream. The stereo matching process employs a semi-global stereo matching algorithm. First, distortion correction and epipolar correction are performed on the left-eye and right-eye views to eliminate lens distortion and line shift of the binocular cameras. Then, the disparity map between the corrected left and right views is calculated. Based on the disparity map and the intrinsic parameter matrix of the binocular cameras, the 3D spatial coordinates corresponding to each pixel are calculated to generate 3D point cloud data containing depth information. Each point in the 3D point cloud data corresponds to a pixel in the image, containing the 3D spatial coordinates and color information of that pixel. This accurately reflects the 3D spatial position and facial detail features of students' faces in the classroom, providing accurate 3D data support for subsequent facial muscle movement unit feature extraction.

[0083] The processing unit synchronizes the clocks of the environmental sensor and the binocular camera using the Network Time Protocol (NTP), assigning a unified timestamp to each frame of the light intensity, color temperature data, and video stream. Specifically, the processing unit acts as the server for the NTP, while the binocular camera and environmental sensor act as clients. The clients periodically send clock synchronization requests to the server, which returns the current standard clock time. The clients adjust their local clocks according to the returned standard clock time to ensure that the local clocks of the binocular camera and environmental sensor are synchronized with the standard clock of the processing unit. The clock synchronization error is lower than a preset synchronization error threshold, preferably 1 millisecond.

[0084] After clock synchronization is complete, the binocular camera adds a unified timestamp to each frame of the acquired video stream, corresponding to the acquisition time. The environmental sensor adds a unified timestamp to each set of light intensity and color temperature data, corresponding to the acquisition time, ensuring that the time axis of the video stream data and the environmental parameter data are completely consistent, providing a unified time reference for subsequent feature weighting correction and time alignment.

[0085] In this embodiment, the course scheduling optimization system based on micro-expression information includes a data acquisition device, a feature extraction device, a stress calculation device, and a scheduling engine device. The data acquisition device is communicatively connected to a binocular camera and an environmental sensor deployed in the classroom to acquire video streams captured by the binocular camera and light intensity and color temperature data collected by the environmental sensor. The output of the data acquisition device is connected to the input of the feature extraction device.

[0086] The feature extraction device incorporates a spatiotemporal graph convolutional network model. It inputs video streams along with illumination intensity and color temperature data into the spatiotemporal graph convolutional network, using these data as bias terms to weight and correct the node features of the graph convolutional layers. The output is a facial muscle movement unit feature sequence that eliminates environmental interference. The output of the feature extraction device is connected to the input of the stress calculation device. The stress calculation device aligns the facial muscle movement unit feature sequence with course timestamps, calculates the frequency of feature changes within a set time window, and generates a cognitive stress time series curve at the class level. The output of the stress calculation device is connected to the input of the scheduling engine device. The scheduling engine incorporates a multi-objective genetic algorithm scheduling engine. It inputs the cognitive stress time series curve into the multi-objective genetic algorithm scheduling engine, marking time segments where stress peaks exceed a preset threshold as unscheduling penalty items. It solves the problem using teacher resource availability and classroom physical resource availability as boundary conditions to generate a globally optimal course schedule.

[0087] In this embodiment, a multi-objective genetic algorithm scheduling engine is used. The fitness value is obtained by using the weighted sum of the unschedulable course penalty, teacher resource conflict penalty, and classroom resource conflict penalty as the fitness value. The algorithm is iteratively solved under resource constraints to generate a globally optimal course schedule that conforms to the cognitive load pattern of students. High steep increase windows are identified by the pressure change rate, and the probability of mutation operation of the genetic algorithm is adaptively adjusted to improve the convergence speed and solution accuracy of the algorithm. The scheduling adjustment command is generated by comparing the schedule and sent to relevant terminals. Three-dimensional point cloud data is generated by binocular stereo matching, which provides accurate data support and time benchmark for feature extraction and time alignment.

Claims

1. A course scheduling optimization method based on micro-expression information, characterized in that, include: Acquire video streams from binocular cameras deployed in the classroom, as well as light intensity and color temperature data collected by environmental sensors; The video stream and the light intensity and color temperature data are input into a spatiotemporal graph convolutional network. The light intensity and color temperature data are used as bias terms to perform weighted correction on the node features of the graph convolutional layer in the spatiotemporal graph convolutional network, and the facial muscle motion unit feature sequence that eliminates environmental interference is output. The facial muscle movement unit feature sequence is time-aligned with the course timestamp, the frequency of feature changes within a set time window is calculated, and a cognitive stress time series curve in the class dimension is generated. The cognitive stress time series curve is input into the multi-objective genetic algorithm scheduling engine. Time segments where the stress peak exceeds the preset threshold are marked as non-scheduling penalty items. The solution is performed with the teacher resource idle status and the classroom physical resource availability status as boundary conditions to generate a globally optimal course scheduling timetable. 2.The micro-expression information-based course scheduling optimization method of claim 1, wherein, Using the light intensity and color temperature data as bias terms, the node features of the graph convolutional layer in the spatiotemporal graph convolutional network are weighted and corrected, including: The light intensity and color temperature data are converted into normalized environmental feature vectors; In each graph convolution operation of the spatiotemporal graph convolutional network, the normalized environment feature vector is multiplied bitwise with the output feature matrix of the current layer graph convolution kernel to generate an environment-corrected feature matrix. The environmental correction feature matrix is ​​input into the activation function. 3.The micro-expression information-based course scheduling optimization method of claim 1, wherein, Calculate the frequency of feature changes within a set time window to generate a time-series curve of cognitive stress at the class level, including: The activation states of the frowning and blinking action units are extracted from the facial muscle movement unit feature sequence; Within the set time window, the activation frequency of the frowning action unit and the duration of continuous activation of the blinking action unit are counted. Substitute the activation frequency and the continuous activation duration into a preset pressure conversion function to obtain the pressure value corresponding to the set time window. By stitching together the stress values ​​corresponding to each set time window in chronological order, a cognitive stress time-series curve for the class dimension is generated. 4.The micro-expression information-based course scheduling optimization method of claim 1, wherein, The time segments where the pressure peak exceeds a preset threshold are marked as non-scheduling penalty items. The solution is performed using the teacher resource availability status and classroom physical resource availability status as boundary conditions, including: Construct an initial population, where each individual in the initial population represents a course time allocation scheme; Calculate the fitness value for each individual, which is a weighted sum of the unscheduling penalty, the teacher resource conflict penalty, and the classroom resource conflict penalty; When the fitness value meets the convergence condition or the number of iterations reaches the upper limit, the individual with the lowest fitness value is output as the globally optimal course schedule. 5.The micro-expression information-based course scheduling optimization method of claim 1, wherein, Acquire video streams from binocular cameras deployed in the classroom, as well as light intensity and color temperature data from environmental sensors, including: The binocular camera is used to acquire left-eye and right-eye views, and stereo matching of the left-eye and right-eye views is performed to generate three-dimensional point cloud data containing depth information. The three-dimensional point cloud data is used as the video stream. The environmental sensor and the binocular camera are synchronized using the Network Time Protocol (NTP) to assign a unified timestamp to the light intensity and color temperature data, as well as to each frame of the video stream. 6.The micro-expression information-based course scheduling optimization method of claim 1, wherein, After generating the globally optimal course schedule, the following steps are also included: The globally optimal course schedule is compared with the currently executing initial course schedule to extract the target course segments that have undergone time changes. The teaching content identifier, original timestamp, and updated timestamp corresponding to the target course segment are encapsulated into a scheduling adjustment instruction; The scheduling adjustment instruction is sent to the teacher's terminal and the classroom display device for display.

7. The course scheduling optimization method based on micro-expression information according to claim 2, characterized in that, After generating the facial muscle movement unit feature sequence that eliminates environmental interference, the method further includes: Extract the facial region mask from the video stream; When the area of ​​the connected component of the facial region mask is less than a preset area threshold, it is determined that there is local occlusion; In the spatiotemporal graph convolutional network, determine the spatial neighbor nodes corresponding to the occluded nodes; The features of the occluded nodes are aggregated and completed using the environmental correction feature matrix of the spatial neighborhood nodes.

8. The course scheduling optimization method based on micro-expression information according to claim 3, characterized in that, After generating the cognitive stress time-series curve for the class dimension, the following is also included: Acquire data on the teacher's movement trajectory within the classroom; The spatial distance between the teacher and the student's face within the set time window is determined based on the movement trajectory data. When the spatial distance is less than the near-distance interaction threshold, the pressure value corresponding to the set time window is marked as the interaction interference value; The median filtering algorithm is used to replace the interaction interference value in the cognitive stress time series curve with the stress value of the adjacent time window.

9. The course scheduling optimization method based on micro-expression information according to claim 4, characterized in that, After calculating the fitness value for each individual, the following is also included: Calculate the first derivative of the cognitive stress time series curve in each set time window to obtain the stress change rate sequence; The windows in the pressure change rate sequence that are greater than the change rate threshold are identified as high steep increase windows; When performing crossover and mutation operations on the initial population, the mutation probability of the gene segment corresponding to the high steep growth window is increased, while the mutation probability of the gene segment corresponding to other windows is decreased, thereby generating a progeny population.

10. A course scheduling optimization system based on micro-expression information, characterized in that, The course scheduling optimization method based on micro-expression information according to any one of claims 1-9 includes: The data acquisition device is used to acquire video streams captured by binocular cameras deployed in the classroom, as well as light intensity and color temperature data collected by environmental sensors. The feature extraction device is used to input the video stream and the light intensity and color temperature data into the spatiotemporal graph convolutional network, and use the light intensity and color temperature data as a bias term to perform weighted correction on the node features of the graph convolutional layer in the spatiotemporal graph convolutional network, and output a facial muscle motion unit feature sequence that eliminates environmental interference. The stress calculation device is used to time-align the facial muscle movement unit feature sequence with the course timestamp, calculate the frequency of feature changes within a set time window, and generate a cognitive stress time series curve in the class dimension. The scheduling engine device is used to input the cognitive stress time series curve into the multi-objective genetic algorithm scheduling engine, mark the time segment where the stress peak exceeds the preset threshold as the penalty item for not being able to schedule classes, and solve the problem with the teacher resource idle status and the classroom physical resource availability status as boundary conditions to generate the globally optimal course scheduling timetable.

Images