A teaching informationization device regulation method and system based on a multimedia device

Abstract

The application belongs to the technical field of intelligent device regulation and control, and particularly relates to a teaching informationization device regulation and control method and system based on multimedia devices. The method comprises a data acquisition module, a scene recognition module, an attention field modeling module, a device regulation and control decision and an effect evaluation module. The application collects classroom environment, device operation and student behavior data through multi-source perception fusion, recognizes a teaching scene based on a fuzzy C-means clustering algorithm, constructs a classroom space student attention field distribution model, solves device optimal regulation and control parameters through a multi-objective optimization multi-device collaborative regulation and control function and executes the parameters, and realizes regulation and control strategy closed-loop self-optimization through quantitative effect evaluation and Bayesian optimization. The application can dynamically adapt to a teaching scene and a student attention state, realizes multi-device collaborative regulation and control, and has strong adaptability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent device control technology, specifically a method and system for controlling teaching information technology equipment based on multimedia devices. Background Technology

[0002] The current classroom teaching model has expanded from the traditional one-way multimedia lecture to a composite teaching model that integrates multiple scenarios such as group discussions, student presentations, self-study, online and offline blended teaching, and standardized examinations. Different teaching scenarios have very different requirements for classroom lighting environment, sound environment and equipment operation status, and have higher requirements for equipment such as projectors, smart interactive displays or zoned smart speakers.

[0003] The decision-making basis of existing equipment control schemes is mostly based on environmental parameters such as ambient light or noise, as well as operating status data such as equipment brightness and volume. They completely fail to incorporate indicators of teaching effectiveness based on students' classroom behavior and attention status. This means that the control can only achieve the numerical standard of environmental parameters. Furthermore, the current scheme uses the logic of independent control of single devices. Each device, such as projectors, speakers, and lights, adjusts independently based on its own parameter thresholds without taking into account the coupling effect between light field, sound field, and visual field, and cannot perform global collaborative optimization of multiple devices.

[0004] Therefore, a teaching information technology equipment control method and system based on multimedia devices is needed to solve the above problems. Summary of the Invention

[0005] To address the technical problems mentioned in the background, this invention provides a method and system for controlling teaching information technology equipment based on multimedia devices.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] The first aspect of this invention provides a method for controlling teaching information technology equipment based on multimedia devices, the specific steps of which are as follows:

[0008] Step 1: Collect classroom environmental parameter data, multimedia equipment operation status data, and student behavior perception data in real time, and normalize all data to obtain a multi-source perception data vector.

[0009] Step 2: Construct a scene feature vector based on the classroom environment parameter data and multimedia equipment operating status data, calculate the membership degree of the scene feature vector to each preset teaching scene category based on the preset fuzzy C-means clustering algorithm, and generate a teaching scene identifier vector;

[0010] Step 3: Based on the students' facial orientation angle and limb movement frequency in the student behavior perception data, calculate the individual attention intensity of each student, and construct the attention field distribution matrix and overall attention concentration index by spatial diffusion through Gaussian kernel function.

[0011] Step 4: Based on the teaching scenario identifier vector and the attention field distribution matrix, calculate the optimal control parameter vector for each multimedia device using a preset multi-device collaborative control function, and send the optimal control parameter vector to the corresponding multimedia device to perform the control operation;

[0012] Step 5: Based on the changes in student behavior perception data and equipment operation status data before and after regulation, calculate the regulation effect score through a preset regulation effect evaluation function. When the score is lower than a preset threshold, trigger the Bayesian optimization algorithm to update the weight coefficients in the multi-device collaborative regulation function.

[0013] In this application, based on the real-time collection of classroom environmental parameter data, multimedia equipment operating status data, and student behavior perception data in step one, all data are normalized to obtain a multi-source perception data vector. The specific steps are as follows:

[0014] The average light intensity, average ambient sound pressure level, and air temperature in the classroom are collected using light sensors, sound sensors, and temperature and humidity sensors, respectively. Several multimedia devices to be controlled are set in the classroom, including display devices, audio devices, and lighting devices. The brightness and color temperature values ​​of the display devices are collected in real time, as are the volume values ​​of the audio devices and the illuminance values ​​of the lighting devices. The real-time power consumption of the several multimedia devices to be controlled is also obtained.

[0015] Several student seats are set up in the classroom. A two-dimensional Cartesian coordinate system is established with the lower left corner of the front door of the classroom as the origin. The x-axis extends to the right along the width of the classroom, and the y-axis extends forward along the length of the classroom to construct a seat coordinate system. The coordinates of each student's seat are obtained based on the seat coordinate system. The facial orientation angle, body movement frequency, and attendance status of students in each seat area are collected in real time by a panoramic camera. It should be noted that the positive direction of the y-axis is set as 0 degrees, and the counterclockwise direction is the direction of angle increase. The attendance status is 0 for an empty seat and 1 for a seat with a student. All data are processed by Min-Max normalization to obtain a multi-source perception data vector.

[0016] In this application, based on step two, a scene feature vector is constructed according to the classroom environment parameter data and multimedia equipment operating status data. Then, the membership degree of the scene feature vector to each preset teaching scene category is calculated based on a preset fuzzy C-means clustering algorithm to generate a teaching scene identifier vector. The specific steps are as follows:

[0017] Preset teaching scenario categories and classroom historical data are extracted from the database. These categories include multimedia lectures, group discussions, student presentations, self-study, and exams. Historical ambient light intensity, ambient noise, normalized brightness, and normalized volume are extracted from the classroom historical data and integrated into historical scene feature vectors. These feature vectors form a training set. Membership values ​​are randomly generated for each historical sample in the training set using a random initialization method. A membership matrix is ​​constructed based on these membership values. The center vector VC of each cluster is updated using the current membership matrix. The calculation logic is as follows: Where h is the cluster category number, j is the historical sample number, M is the total number of historical samples, U is the membership value, and FS is the scene feature vector; the Euclidean distance from each historical sample to each cluster center is calculated based on the updated center vector, and the membership matrix is ​​updated by the Euclidean distance. The change in cluster centers between the current iteration and the previous iteration is calculated. If the change in cluster centers is less than the preset change in centers, the iteration stops, and the cluster centers are mapped to the corresponding preset teaching scene categories to complete the semantic annotation of the cluster centers;

[0018] Load the cluster center matrix obtained from offline training, calculate the Euclidean distance from the real-time scene feature vector to each cluster center, calculate the real-time membership vector based on the distance, extract the real-time membership component of the current scene to any preset teaching scene category from the real-time membership vector, select the scene category corresponding to the largest real-time membership component as the currently identified teaching scene, and obtain the teaching scene identifier vector.

[0019] In this application, based on step three, the individual attention intensity of each student is calculated according to the student's facial orientation angle and limb movement frequency in the student behavior perception data. Spatial diffusion is then performed using a Gaussian kernel function to construct an attention field distribution matrix and an overall attention concentration index. The specific steps are as follows:

[0020] The teaching area center is set based on a two-dimensional Cartesian coordinate system. The ideal target orientation angle of each student relative to the center of the teaching area is calculated using the coordinates of each student's seat. The calculation logic is as follows: ,in Let i be the student's ID, and let i be the arctangent function in the four quadrants. The difference between the student's actual facial orientation angle and the ideal target orientation angle is used to obtain the orientation deviation angle. The orientation deviation is then mapped to the interval [0,1] using a cosine function to obtain the orientation consistency function. If the orientation consistency function is 1 and the orientation deviation angle is 0, the orientation consistency is considered optimal; otherwise, the orientation consistency is considered worst. The limb movement frequency is nonlinearly normalized using a sigmoid function to obtain the movement activity function. The orientation consistency function and the movement activity function are then linearly weighted and fused to obtain the individual attention intensity of the i-th student.

[0021] The classroom's planar area is divided into several uniform grids, and the center coordinates of these grids are set. A two-dimensional Gaussian kernel function is used to spatially diffuse the individual attention intensity of each student, constructing a continuous spatial attention field distribution function. The calculation logic is as follows: Where xs and ys are the planar coordinates of any point within the classroom space. Let be the spatial diffusion coefficient of attention, exp be the natural exponential function, As be the individual attention intensity of the student, s be the total number of effective students in the classroom, and i be the student's ID. The attention field distribution function in the continuous space is discretized and sampled at the center point of the grid to obtain the attention field distribution matrix. Based on the attention field distribution matrix, the overall attention concentration index CY of the classroom is calculated. The calculation logic is as follows: ,in The arithmetic mean of all elements in the attention field distribution matrix. Let qc be the attention field intensity value of any grid in the attention field distribution matrix, pc be the row index of the attention field distribution matrix, M be the total number of rows in the classroom space grid, and N be the total number of columns in the classroom space grid.

[0022] In this application, based on step four, according to the teaching scenario identifier vector and the attention field distribution matrix, the optimal control parameter vector of each multimedia device is calculated through a preset multi-device collaborative control function, and the optimal control parameter vector is sent to the corresponding multimedia device to perform the control operation. The specific steps are as follows:

[0023] Based on the parameters of the multimedia devices to be controlled, a control parameter vector is constructed. The control parameter vector includes the brightness adjustment coefficient, color temperature adjustment coefficient, contrast adjustment coefficient of the projector, the main volume value of each speaker, sound field equalization parameter, left and right channel balance coefficient, and illuminance adjustment coefficient of each zone's intelligent lights. The optimal reference parameter vector corresponding to each scene in the database is extracted, and a scene reference parameter matrix is ​​constructed. All reference parameters satisfy the value constraint of [0,1]. The teaching scene identifier vector and the scene reference parameter matrix are aligned by row number, and then the attention field distribution matrix is ​​globally normalized to obtain the normalized attention field matrix.

[0024] Broadcast the current control parameter vector to the same dimension as the scene reference parameter matrix. Subtract the current control parameter vector from the scene reference parameter matrix to obtain the parameter deviation matrix. Calculate the square of the L2 norm for each row of the parameter deviation matrix to obtain a C-dimensional column vector. Perform a weighted sum of the teaching scene identifier vector and the deviation vector to obtain the scene matching loss. The control parameter vector is decomposed to obtain parameter sub-vectors for lighting, audio, and display devices. Then, based on the lighting installation coordinates and using a two-dimensional Gaussian kernel matrix, a lighting light field sub-matrix is ​​constructed. Based on the speaker installation coordinates and using a sound field attenuation model, an audio sound field sub-matrix is ​​constructed. Based on the screen position and using a visually effective viewing angle model, a display visual field sub-matrix is ​​constructed. The device action field matrix is ​​obtained using the two-dimensional Gaussian kernel matrix, the audio sound field sub-matrix, the visual field matrix, and the parameter sub-vectors of the lighting, audio, and display devices. This device action field matrix is ​​then globally normalized, and the attention field matrix is ​​obtained through normalization. and normalized device action field matrix Calculate the matrix-based KL divergence Its calculation logic is as follows: ,in The KL divergence is used; the energy consumption weights of each preset device parameter in the database are extracted, and an energy consumption weight column vector is constructed. The energy consumption control loss is obtained by calculating the weighted sum of squares through the energy consumption weight column vector and the dot product of the vectors. ;

[0025] The multi-device collaborative control function LH is constructed by using scene matching loss, matrix-based KL divergence, energy consumption control loss, control parameter vector HJ, teaching scene identifier vector US, and attention field distribution matrix YA. Its calculation logic is as follows: ,in , and The preset weight coefficients are used; the optimal control parameter vector is obtained by minimizing the multi-device collaborative control function using the gradient descent method with the current control parameter vector. The optimal control parameter vector is then split according to the device type and mapped to the actual physical control value of each device. The actual physical control value is then sent to the corresponding multimedia device to perform the control operation.

[0026] In this application, based on step five, according to the changes in student behavior perception data and equipment operating status data before and after regulation, a regulation effect score is calculated through a preset regulation effect evaluation function. When the score is lower than a preset threshold, a Bayesian optimization algorithm is triggered to update the weight coefficients in the multi-device collaborative regulation function. The specific steps are as follows:

[0027] Before the implementation of regulation, stable data is obtained as baseline data. The average baseline attention concentration, average baseline attention field matrix, and average baseline total system energy consumption of the baseline data are calculated. Then, stable data after the implementation of regulation is obtained as evaluation data. The average evaluation attention concentration, average evaluation attention field matrix, and average evaluation total system energy consumption of the evaluation data are calculated. The difference between the average evaluation attention concentration and the average baseline attention concentration is used to obtain the concentration deviation. The concentration deviation is divided by the average baseline attention concentration to obtain the rate of change of attention concentration. The center of the teaching area is set as the center of a Gaussian distribution, and a two-dimensional Gaussian distribution function is constructed. The Gaussian distribution function is discretized and sampled at the center point of the classroom grid to obtain the initial ideal matrix. The initial ideal matrix is ​​globally normalized to obtain the ideal attention field distribution matrix. The Manhattan distance before regulation is calculated using the average baseline attention field matrix and the ideal attention field distribution matrix. Then, the Manhattan distance after regulation is calculated using the average evaluation attention field matrix and the ideal attention field distribution matrix. Based on the Manhattan distance before regulation... Distance from Manhattan after regulation The matching improvement rate RJ is calculated using the following logic: Then, based on the average baseline system total energy consumption RF and the average evaluation system total energy consumption RY, the relative change rate HY of the system's total energy consumption before and after regulation is calculated. The calculation logic is as follows: The rate of change in attention concentration, the rate of improvement in matching degree, and the relative rate of change in total energy consumption are linearly weighted and fused to obtain a comprehensive control effect score. If the comprehensive control effect score is greater than or equal to the preset score threshold, the control effect is judged to be excellent; otherwise, the Bayesian optimization process is triggered.

[0028] The Bayesian optimization process is as follows: The current weights and the overall control effect score are added to the historical optimization dataset. If the historical optimization dataset is empty, initial sampling points are generated within the feasible region using Latin hypercube sampling. The weights corresponding to each sampling point are then substituted into the above steps for verification and control, and the corresponding score is calculated. If the dataset is not empty, existing historical data is used directly. The optimal score of the current historical optimization dataset is obtained, and the next sampling point is obtained through the expected improvement of the sampling function and the optimal score. The sampling point is converted into a weight coefficient, substituted into the multi-device collaborative control function, and the corresponding control effect score is obtained. The new sampling data is added to the historical optimization dataset. From the updated historical optimization dataset, the sampling point with the highest score is selected and converted into the optimal weight coefficient.

[0029] A second aspect of this invention provides a teaching information technology equipment control system based on multimedia devices, comprising a data acquisition module, a scene recognition module, an attention field modeling module, an equipment control decision module, and an effect evaluation module.

[0030] The data acquisition module is used to collect classroom environmental parameter data, multimedia equipment operating status data, and student behavior perception data in real time.

[0031] The scene recognition module is used to construct a scene feature vector based on the classroom environment parameter data and multimedia equipment operation status data, calculate the membership degree of the scene feature vector to each preset teaching scene category based on a preset fuzzy C-means clustering algorithm, and generate a teaching scene identifier vector.

[0032] The attention field modeling module calculates the individual attention intensity of each student based on the student's facial orientation angle and limb movement frequency in the student behavior perception data, and constructs the attention field distribution matrix and overall attention concentration index by spatial diffusion through Gaussian kernel function.

[0033] The device control decision module is used to calculate the optimal control parameter vector of each multimedia device based on the teaching scenario identifier vector and the attention field distribution matrix, through a preset multi-device collaborative control function, and send the optimal control parameter vector to the corresponding multimedia device to perform control operation;

[0034] The effect evaluation module is used to calculate the control effect score based on the changes in the operating status data of multimedia equipment before and after the control and the changes in the students' behavioral perception data, and to generate feedback parameters for optimizing the control strategy.

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] The fuzzy C-means clustering algorithm is used to achieve soft classification and recognition of teaching scenarios. The membership vector fully represents the degree of belonging of the current scenario to preset scenarios such as multimedia lectures and group discussions. It can not only identify teaching scenarios, but also smoothly adapt to the transition and switching of scenarios during the teaching process, remove the jump of equipment parameters when switching scenarios, and improve the accuracy of scenario recognition.

[0037] Using the spatial distribution characteristics of students' attention as the basis for equipment control decisions, the intensity of individual students' attention is accurately quantified by weighted fusion of the orientation consistency function and the action activity function. Then, the attention space is diffused and modeled by a two-dimensional Gaussian kernel function, transforming the discrete individual attention characteristics into a continuously distributed attention field throughout the classroom, thus completing the spatial representation of students' attention state. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The following drawings are not drawn to scale according to the actual size, but are intended to illustrate the main idea of ​​the present invention.

[0039] Figure 1 This is a diagram illustrating the method steps of the present invention.

[0040] Figure 2 This is a system module connection diagram of the present invention. Detailed Implementation

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are also within the scope of protection of the present invention.

[0042] Please refer to Figure 1 As shown, the first aspect of the present invention provides a method for controlling teaching information technology equipment based on multimedia devices, the specific steps of which are as follows:

[0043] Step 1: Collect classroom environmental parameter data, multimedia equipment operation status data, and student behavior perception data in real time, and normalize all data to obtain a multi-source perception data vector.

[0044] Step 2: Construct a scene feature vector based on the classroom environment parameter data and multimedia equipment operating status data, calculate the membership degree of the scene feature vector to each preset teaching scene category based on the preset fuzzy C-means clustering algorithm, and generate a teaching scene identifier vector;

[0045] Step 3: Based on the student's facial orientation angle and body movement frequency in the student behavior perception data, calculate the individual attention intensity of each student, and construct the attention field distribution matrix and overall attention concentration index by performing spatial diffusion through the Gaussian kernel function.

[0046] Step 4: Based on the teaching scenario identifier vector and the attention field distribution matrix, calculate the optimal control parameter vector for each multimedia device using a preset multi-device collaborative control function, and send the optimal control parameter vector to the corresponding multimedia device to perform the control operation;

[0047] Step 5: Based on the changes in student behavior perception data and equipment operation status data before and after regulation, calculate the regulation effect score through a preset regulation effect evaluation function. When the score is lower than a preset threshold, trigger the Bayesian optimization algorithm to update the weight coefficients in the multi-device collaborative regulation function.

[0048] In this application, based on the real-time collection of classroom environmental parameter data, multimedia equipment operating status data, and student behavior perception data in step one, all data are normalized to obtain a multi-source perception data vector. The specific steps are as follows:

[0049] The average light intensity, average ambient sound pressure level, and air temperature in the classroom are collected using light sensors, sound sensors, and temperature and humidity sensors, respectively. Several multimedia devices to be controlled are set in the classroom, including display devices, audio devices, and lighting devices. The brightness and color temperature values ​​of the display devices are collected in real time, as are the volume values ​​of the audio devices and the illuminance values ​​of the lighting devices. The real-time power consumption of the several multimedia devices to be controlled is also obtained.

[0050] Several student seats are set up in the classroom. A two-dimensional Cartesian coordinate system is established with the lower left corner of the front door of the classroom as the origin. The x-axis extends to the right along the width of the classroom, and the y-axis extends forward along the length of the classroom to construct a seat coordinate system. The coordinates of each student's seat are obtained based on the seat coordinate system. The facial orientation angle, body movement frequency, and attendance status of students in each seat area are collected in real time by a panoramic camera. It should be noted that the positive direction of the y-axis is set as 0 degrees, and the counterclockwise direction is the direction of angle increase. The attendance status is 0 for an empty seat and 1 for a seat with a student. All data are processed by Min-Max normalization to obtain a multi-source perception data vector.

[0051] In this application, based on step two, a scene feature vector is constructed according to the classroom environment parameter data and multimedia equipment operating status data. Then, the membership degree of the scene feature vector to each preset teaching scene category is calculated based on a preset fuzzy C-means clustering algorithm to generate a teaching scene identifier vector. The specific steps are as follows:

[0052] Preset teaching scenario categories and classroom historical data are extracted from the database. These categories include multimedia lectures, group discussions, student presentations, self-study, and exams. Historical ambient light intensity, ambient noise, normalized brightness, and normalized volume are extracted from the classroom historical data and integrated into historical scene feature vectors. These feature vectors form a training set. Membership values ​​are randomly generated for each historical sample in the training set using a random initialization method. A membership matrix is ​​constructed based on these membership values. The center vector VC of each cluster is updated using the current membership matrix. The calculation logic is as follows: Where h is the cluster category number, j is the historical sample number, M is the total number of historical samples, U is the membership value, and FS is the scene feature vector; the Euclidean distance from each historical sample to each cluster center is calculated based on the updated center vector, and the membership matrix is ​​updated by the Euclidean distance. The change in cluster centers between the current iteration and the previous iteration is calculated. If the change in cluster centers is less than the preset change in centers, the iteration is stopped, and the cluster centers are mapped to the corresponding preset teaching scene categories to complete the semantic annotation of the cluster centers;

[0053] Load the cluster center matrix obtained from offline training, calculate the Euclidean distance from the real-time scene feature vector to each cluster center, calculate the real-time membership vector based on the distance, extract the real-time membership component of the current scene to any preset teaching scene category from the real-time membership vector, select the scene category corresponding to the largest real-time membership component as the currently identified teaching scene, and obtain the teaching scene identifier vector.

[0054] In this application, based on step three, the individual attention intensity of each student is calculated according to the student's facial orientation angle and limb movement frequency in the student behavior perception data. Spatial diffusion is then performed using a Gaussian kernel function to construct an attention field distribution matrix and an overall attention concentration index. The specific steps are as follows:

[0055] The teaching area center is set based on a two-dimensional Cartesian coordinate system. The ideal target orientation angle of each student relative to the center of the teaching area is calculated using the coordinates of each student's seat. The calculation logic is as follows: ,in Let i be the student's ID, and let i be the arctangent function in the four quadrants. The difference between the student's actual facial orientation angle and the ideal target orientation angle is used to obtain the orientation deviation angle. The orientation deviation is then mapped to the interval [0,1] using a cosine function to obtain the orientation consistency function. If the orientation consistency function is 1 and the orientation deviation angle is 0, the orientation consistency is considered optimal; otherwise, the orientation consistency is considered worst. The limb movement frequency is nonlinearly normalized using a sigmoid function to obtain the movement activity function. The orientation consistency function and the movement activity function are then linearly weighted and fused to obtain the individual attention intensity of the i-th student.

[0056] The classroom's planar area is divided into several uniform grids, and the center coordinates of these grids are set. A two-dimensional Gaussian kernel function is used to spatially diffuse the individual attention intensity of each student, constructing a continuous spatial attention field distribution function. The calculation logic is as follows: Where xs and ys are the planar coordinates of any point within the classroom space. Let be the attention spatial diffusion coefficient, exp be the natural exponential function, and As i Let be the individual attention intensity corresponding to student i, s be the total number of effective students in the classroom, and i be the student's ID; the attention field distribution function in the continuous space is discretized and sampled at the center point of the grid to obtain the attention field distribution matrix. Based on the attention field distribution matrix, the overall attention concentration index CY of the classroom is calculated, and its calculation logic is as follows: ,in The arithmetic mean of all elements in the attention field distribution matrix. Let qc be the attention field intensity value of any grid in the attention field distribution matrix, pc be the row index of the attention field distribution matrix, M be the total number of rows in the classroom space grid, and N be the total number of columns in the classroom space grid.

[0057] In this application, based on step four, according to the teaching scenario identifier vector and the attention field distribution matrix, the optimal control parameter vector of each multimedia device is calculated through a preset multi-device collaborative control function, and the optimal control parameter vector is sent to the corresponding multimedia device to perform the control operation. The specific steps are as follows:

[0058] Based on the parameters of the multimedia devices to be controlled, a control parameter vector is constructed. The control parameter vector includes the brightness adjustment coefficient, color temperature adjustment coefficient, contrast adjustment coefficient of the projector, the main volume value of each speaker, sound field equalization parameter, left and right channel balance coefficient, and illuminance adjustment coefficient of each zone's intelligent lights. The optimal reference parameter vector corresponding to each scene in the database is extracted, and a scene reference parameter matrix is ​​constructed. All reference parameters satisfy the value constraint of [0,1]. The teaching scene identifier vector and the scene reference parameter matrix are aligned by row number, and then the attention field distribution matrix is ​​globally normalized to obtain the normalized attention field matrix.

[0059] Broadcast the current control parameter vector to the same dimension as the scene reference parameter matrix. Subtract the current control parameter vector from the scene reference parameter matrix to obtain the parameter deviation matrix. Calculate the square of the L2 norm for each row of the parameter deviation matrix to obtain a C-dimensional column vector. Perform a weighted sum of the teaching scene identifier vector and the deviation vector to obtain the scene matching loss. The control parameter vector is decomposed to obtain parameter sub-vectors for lighting, audio, and display devices. Then, based on the lighting installation coordinates and using a two-dimensional Gaussian kernel matrix, a lighting light field sub-matrix is ​​constructed. Based on the speaker installation coordinates and using a sound field attenuation model, an audio sound field sub-matrix is ​​constructed. Based on the screen position and using a visually effective viewing angle model, a display visual field sub-matrix is ​​constructed. The device action field matrix is ​​obtained using the two-dimensional Gaussian kernel matrix, the audio sound field sub-matrix, the visual field matrix, and the parameter sub-vectors of the lighting, audio, and display devices. This device action field matrix is ​​then globally normalized, and the attention field matrix is ​​obtained through normalization. and normalized device action field matrix Calculate the matrix-based KL divergence Its calculation logic is as follows: ,in The KL divergence is used; the energy consumption weights of each preset device parameter in the database are extracted, and an energy consumption weight column vector is constructed. The energy consumption control loss is obtained by calculating the weighted sum of squares through the energy consumption weight column vector and the dot product of the vectors. ;

[0060] The multi-device collaborative control function LH is constructed by using scene matching loss, matrix-based KL divergence, energy consumption control loss, control parameter vector HJ, teaching scene identifier vector US, and attention field distribution matrix YA. Its calculation logic is as follows: ,in , and The preset weight coefficients are used; the optimal control parameter vector is obtained by minimizing the multi-device collaborative control function using the gradient descent method with the current control parameter vector. The optimal control parameter vector is then split according to the device type and mapped to the actual physical control value of each device. The actual physical control value is then sent to the corresponding multimedia device to perform the control operation.

[0061] In this application, based on step five, according to the changes in student behavior perception data and equipment operating status data before and after regulation, a regulation effect score is calculated through a preset regulation effect evaluation function. When the score is lower than a preset threshold, a Bayesian optimization algorithm is triggered to update the weight coefficients in the multi-device collaborative regulation function. The specific steps are as follows:

[0062] Before the implementation of regulation, stable data is obtained as baseline data. The average baseline attention concentration, average baseline attention field matrix, and average baseline total system energy consumption of the baseline data are calculated. Then, stable data after the implementation of regulation is obtained as evaluation data. The average evaluation attention concentration, average evaluation attention field matrix, and average evaluation total system energy consumption of the evaluation data are calculated. The difference between the average evaluation attention concentration and the average baseline attention concentration is used to obtain the concentration deviation. The concentration deviation is divided by the average baseline attention concentration to obtain the rate of change of attention concentration. The center of the teaching area is set as the center of a Gaussian distribution, and a two-dimensional Gaussian distribution function is constructed. The Gaussian distribution function is discretized and sampled at the center point of the classroom grid to obtain the initial ideal matrix. The initial ideal matrix is ​​globally normalized to obtain the ideal attention field distribution matrix. The Manhattan distance before regulation is calculated using the average baseline attention field matrix and the ideal attention field distribution matrix. Then, the Manhattan distance after regulation is calculated using the average evaluation attention field matrix and the ideal attention field distribution matrix. Based on the Manhattan distance before regulation... Distance from Manhattan after regulation The matching improvement rate RJ is calculated using the following logic: Then, based on the average baseline system total energy consumption RF and the average evaluation system total energy consumption RY, the relative change rate HY of the system's total energy consumption before and after regulation is calculated. The calculation logic is as follows: The rate of change in attention concentration, the rate of improvement in matching degree, and the relative rate of change in total energy consumption are linearly weighted and fused to obtain a comprehensive control effect score. If the comprehensive control effect score is greater than or equal to the preset score threshold, the control effect is judged to be excellent; otherwise, the Bayesian optimization process is triggered.

[0063] The Bayesian optimization process is as follows: The current weights and the overall control effect score are added to the historical optimization dataset. If the historical optimization dataset is empty, initial sampling points are generated within the feasible region using Latin hypercube sampling. The weights corresponding to each sampling point are then substituted into the above steps for verification and control, and the corresponding score is calculated. If the dataset is not empty, existing historical data is used directly. The optimal score of the current historical optimization dataset is obtained, and the next sampling point is obtained through the expected improvement of the sampling function and the optimal score. The sampling point is converted into a weight coefficient, substituted into the multi-device collaborative control function, and the corresponding control effect score is obtained. The new sampling data is added to the historical optimization dataset. From the updated historical optimization dataset, the sampling point with the highest score is selected and converted into the optimal weight coefficient.

[0064] Please refer to Figure 2 As shown, a second aspect of the present invention provides a teaching information technology equipment control system based on multimedia devices, including a data acquisition module, a scene recognition module, an attention field modeling module, an equipment control decision module, and an effect evaluation module:

[0065] The data acquisition module is used to collect classroom environmental parameter data, multimedia equipment operating status data, and student behavior perception data in real time.

[0066] The scene recognition module is used to construct a scene feature vector based on the classroom environment parameter data and multimedia equipment operation status data, calculate the membership degree of the scene feature vector to each preset teaching scene category based on a preset fuzzy C-means clustering algorithm, and generate a teaching scene identifier vector.

[0067] The attention field modeling module calculates the individual attention intensity of each student based on the student's facial orientation angle and limb movement frequency in the student behavior perception data, and constructs the attention field distribution matrix and overall attention concentration index by spatial diffusion through Gaussian kernel function.

[0068] The device control decision module is used to calculate the optimal control parameter vector of each multimedia device based on the teaching scenario identifier vector and the attention field distribution matrix, through a preset multi-device collaborative control function, and send the optimal control parameter vector to the corresponding multimedia device to perform control operation;

[0069] The effect evaluation module is used to calculate the control effect score based on the changes in the operating status data of multimedia equipment before and after the control and the changes in the students' behavioral perception data, and to generate feedback parameters for optimizing the control strategy.

[0070] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, ATA hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state ATA hard disk.

[0071] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0072] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0073] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0074] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0075] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0076] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable ATA hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0077] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for controlling teaching information technology equipment based on multimedia devices, characterized in that, Includes the following steps: Step 1: Collect classroom environmental parameter data, multimedia equipment operation status data, and student behavior perception data in real time, and normalize all data to obtain multi-source perception data vectors. Step 2: Construct a scene feature vector based on the classroom environment parameter data and multimedia equipment operation status data. Calculate the membership degree of the scene feature vector to each preset teaching scene category based on a preset fuzzy C-means clustering algorithm to generate a teaching scene identifier vector. Step 3: Based on the students' facial orientation angle and limb movement frequency in the student behavior perception data, calculate the individual attention intensity of each student, and construct the attention field distribution matrix and overall attention concentration index by spatial diffusion through Gaussian kernel function. Step 4: Based on the teaching scenario identifier vector and the attention field distribution matrix, calculate the optimal control parameter vector for each multimedia device using a preset multi-device collaborative control function, and send the optimal control parameter vector to the corresponding multimedia device to perform the control operation; Step 5: Based on the changes in student behavior perception data and equipment operation status data before and after regulation, calculate the regulation effect score through a preset regulation effect evaluation function. When the score is lower than a preset threshold, trigger the Bayesian optimization algorithm to update the weight coefficients in the multi-device collaborative regulation function.

2. The method for controlling teaching information technology equipment based on multimedia devices according to claim 1, characterized in that, The specific steps to obtain the multi-source sensing data vector are as follows: The average light intensity, average ambient sound pressure level, and air temperature in the classroom are collected using light sensors, sound sensors, and temperature and humidity sensors, respectively. Several multimedia devices to be controlled are set in the classroom, including display devices, audio devices, and lighting devices. The brightness and color temperature values ​​of the display devices are collected in real time, as are the volume values ​​of the audio devices and the illuminance values ​​of the lighting devices. The real-time power consumption of the several multimedia devices to be controlled is also obtained. Several student seats are set up in the classroom. A two-dimensional Cartesian coordinate system is established with the lower left corner of the front door of the classroom as the origin. The x-axis extends to the right along the width of the classroom, and the y-axis extends forward along the length of the classroom to construct a seat coordinate system. The coordinates of each student's seat are obtained based on the seat coordinate system. The facial orientation angle, limb movement frequency, and attendance status of students in each seat area are collected in real time by a panoramic camera. All data are processed by Min-Max normalization to obtain a multi-source perception data vector.

3. The method for controlling teaching information technology equipment based on multimedia devices according to claim 1, characterized in that, The specific steps for generating teaching scenario identifier vectors are as follows: The system extracts preset teaching scenario categories and classroom historical data from the database. The preset teaching scenario categories include multimedia lecture scenarios, group discussion scenarios, student presentation scenarios, self-study scenarios, and examination scenarios. It extracts historical ambient light intensity, historical ambient noise, historical normalized brightness value, and historical normalized volume value from the classroom historical data and integrates them into historical scenario feature vectors. The historical scenario feature vectors form a training set. Through random initialization, several membership values ​​are randomly generated for each historical sample in the training set. A membership matrix is ​​constructed based on several membership values. The center vector of each cluster is updated through the current membership matrix. Based on the updated center vector, the Euclidean distance from each historical sample to each cluster center is calculated. The membership matrix is ​​then updated through the Euclidean distance, and the change in cluster centers between the current iteration and the previous iteration is calculated. If the change in cluster centers is less than the preset change in centers, the iteration stops, and the cluster centers are mapped to the corresponding preset teaching scenario categories to complete the semantic annotation of the cluster centers. Load the cluster center matrix obtained from offline training, calculate the Euclidean distance from the real-time scene feature vector to each cluster center, calculate the real-time membership vector based on the distance, extract the real-time membership component of the current scene to any preset teaching scene category from the real-time membership vector, select the scene category corresponding to the largest real-time membership component as the currently identified teaching scene, and obtain the teaching scene identifier vector.

4. The method for controlling teaching information technology equipment based on multimedia devices according to claim 1, characterized in that, Based on the students' facial orientation angles and body movement frequencies from the student behavioral perception data, the individual attention intensity of each student is calculated, and spatial diffusion is performed using a Gaussian kernel function to construct an attention field distribution matrix and an overall attention concentration index. The specific steps are as follows: The teaching area center is set based on a two-dimensional Cartesian coordinate system. The ideal target orientation angle of each student relative to the center of the teaching area is calculated using the coordinates of each student's seat. The difference between the student's actual facial orientation angle and the ideal target orientation angle is used to obtain the orientation deviation angle. The orientation deviation is mapped to the [0,1] interval using a cosine function to obtain the orientation consistency function. The frequency of limb movements is nonlinearly normalized using a sigmoid function to obtain the movement activity function. The orientation consistency function and the movement activity function are linearly weighted and fused to obtain the individual attention intensity of the corresponding student. The classroom's planar area is divided into several uniform grids, and the center coordinates of these grids are set. The individual attention intensity of each student is spatially diffused using a two-dimensional Gaussian kernel function to construct a continuous spatial attention field distribution function. The continuous spatial attention field distribution function is then discretized and sampled at the grid center points to obtain the attention field distribution matrix. Based on the attention field distribution matrix, the overall attention concentration index of the classroom is calculated.

5. The method for controlling teaching information technology equipment based on multimedia devices according to claim 1, characterized in that, Based on the teaching scenario identifier vector and the attention field distribution matrix, the optimal control parameter vector for each multimedia device is calculated using a preset multi-device collaborative control function. The optimal control parameter vector is then sent to the corresponding multimedia device to perform the control operation. The specific steps are as follows: Based on the parameters of the multimedia devices to be controlled, a control parameter vector is constructed. The control parameter vector includes the brightness adjustment coefficient, color temperature adjustment coefficient, contrast adjustment coefficient of the projector, the main volume value of each speaker, sound field equalization parameter, left and right channel balance coefficient, and illuminance adjustment coefficient of each zone's intelligent lights. The optimal reference parameter vector corresponding to each scene in the database is extracted, and a scene reference parameter matrix is ​​constructed. All reference parameters satisfy the value constraint of [0,1]. The teaching scene identifier vector and the scene reference parameter matrix are aligned by row number, and then the attention field distribution matrix is ​​globally normalized to obtain the normalized attention field matrix. Broadcast the current control parameter vector to the same dimension as the scene reference parameter matrix. Subtract the current control parameter vector from the scene reference parameter matrix to obtain the parameter deviation matrix. Calculate the square of the L2 norm for each row of the parameter deviation matrix to obtain a C-dimensional column vector. Sum the teaching scene identifier vector and the deviation vector with weights to obtain the scene matching loss. The control parameter vector is decomposed to obtain parameter sub-vectors for lighting, audio, and display devices. Then, a lighting light field sub-matrix is ​​constructed based on the lighting installation coordinates and a two-dimensional Gaussian kernel matrix. An audio sound field sub-matrix is ​​constructed based on the speaker installation coordinates and a sound field attenuation model. A display visual field sub-matrix is ​​constructed based on the screen position and a visually effective viewing angle model. The device action field matrix is ​​obtained through the two-dimensional Gaussian kernel matrix, the audio sound field sub-matrix, the visual field sub-matrix, and the parameter sub-vectors of lighting, audio, and display devices. The device action field matrix is ​​globally normalized. The matrix-based KL divergence is calculated through the normalized attention field matrix and the normalized device action field matrix. Extract the energy consumption weights of each device parameter from the database, construct an energy consumption weight column vector, and calculate the weighted sum of squares by the energy consumption weight column vector and the inner product of the vectors to obtain the energy consumption control loss. A multi-device collaborative control function is constructed by using scene matching loss, matrix-based KL divergence, energy consumption control loss, control parameter vector, teaching scene identifier vector, and attention field distribution matrix. The multi-device collaborative control function is minimized using the gradient descent method with the current control parameter vector to obtain the optimal control parameter vector. The optimal control parameter vector is then split according to device type and mapped to the actual physical control value of each device. The actual physical control value is then sent to the corresponding multimedia device to perform the control operation.

6. The method for controlling teaching information technology equipment based on multimedia devices according to claim 1, characterized in that, Based on the changes in student behavior perception data and equipment operation status data before and after the adjustment, the adjustment effect score is calculated using a preset adjustment effect evaluation function. The specific steps are as follows: Before the implementation of regulation, stable data is obtained as baseline data. The average baseline attention concentration, average baseline attention field matrix, and average baseline total system energy consumption of the baseline data are calculated. Then, stable data after the implementation of regulation is obtained as evaluation data. The average evaluation attention concentration, average evaluation attention field matrix, and average evaluation total system energy consumption of the evaluation data are calculated. The difference between the average evaluation attention concentration and the average baseline attention concentration is used to obtain the concentration deviation. The concentration deviation is divided by the average baseline attention concentration to obtain the rate of change of attention concentration. The center of the teaching area is set as the center of a Gaussian distribution, and a two-dimensional Gaussian distribution function is constructed. The Gaussian distribution function is discretized and sampled at the center point of the classroom grid to obtain the initial ideal matrix. The ideal matrix is ​​globally normalized to obtain the ideal attention field distribution matrix. The Manhattan distance before regulation is calculated using the average baseline attention field matrix and the ideal attention field distribution matrix. Then, the Manhattan distance after regulation is calculated using the average evaluation attention field matrix and the ideal attention field distribution matrix. The matching degree improvement rate is calculated based on the Manhattan distance before and after regulation. The relative change rate of total energy consumption of the system before and after regulation is calculated based on the average baseline system total energy consumption and the average evaluation system total energy consumption. The change rate of attention concentration, the matching degree improvement rate, and the relative change rate of total energy consumption are linearly weighted and fused to obtain a comprehensive regulation effect score. If the comprehensive regulation effect score is greater than or equal to the preset score threshold, the regulation effect is judged to be excellent; otherwise, the Bayesian optimization process is triggered.

7. The method for controlling teaching information technology equipment based on multimedia devices according to claim 6, characterized in that, When the score is lower than a preset threshold, a Bayesian optimization algorithm is triggered to update the weight coefficients in the multi-device collaborative control function. The Bayesian optimization process is as follows: The current weights and comprehensive control effect scores are added to the historical optimization dataset. If the historical optimization dataset is empty, initial sampling points are generated in the feasible region through Latin hypercube sampling. The weights corresponding to each sampling point are substituted into the above steps to verify the control and calculate the corresponding scores. If the dataset is not empty, existing historical data is used directly. The optimal score of the current historical optimization dataset is obtained, and the next sampling point is obtained through the expected improvement sampling function and the optimal score. The sampling point is converted into a weight coefficient, which is substituted into the multi-device collaborative control function to obtain the corresponding control effect score. The new sampling data is added to the historical optimization dataset. From the updated historical optimization dataset, the sampling point with the highest score is selected and converted into the optimal weight coefficient.

8. A teaching information technology equipment control system based on multimedia devices, implementing the method of any one of claims 1-7, characterized in that, It includes a data acquisition module, a scene recognition module, an attention field modeling module, a device control decision module, and an effect evaluation module. The data acquisition module is used to collect classroom environmental parameter data, multimedia equipment operating status data, and student behavior perception data in real time. The scene recognition module is used to construct a scene feature vector based on the classroom environment parameter data and multimedia equipment operation status data, calculate the membership degree of the scene feature vector to each preset teaching scene category based on a preset fuzzy C-means clustering algorithm, and generate a teaching scene identifier vector. The attention field modeling module calculates the individual attention intensity of each student based on the student's facial orientation angle and limb movement frequency in the student behavior perception data, and constructs the attention field distribution matrix and overall attention concentration index by spatial diffusion through Gaussian kernel function. The device control decision module is used to calculate the optimal control parameter vector of each multimedia device based on the teaching scenario identifier vector and the attention field distribution matrix, through a preset multi-device collaborative control function, and send the optimal control parameter vector to the corresponding multimedia device to perform control operation; The effect evaluation module is used to calculate the control effect score based on the changes in the operating status data of multimedia equipment before and after the control and the changes in the students' behavioral perception data, and to generate feedback parameters for optimizing the control strategy.

Images