Intelligent blackboard synchronous memory display method and system

Abstract

The invention discloses an intelligent blackboard synchronous memory display method and system, and relates to the technical field of intelligent blackboards. An intelligent blackboard synchronous memory display system comprises an intelligent blackboard display adjusting module, an intelligent blackboard synchronous memory module and an intelligent blackboard starting control module. According to the invention, through real-time analysis of environment illumination, sight distance and teacher gestures, display parameters are dynamically adjusted so as to optimize visual experience; by using a display adjustment model constructed by fusion of CNN and MLP models and a gesture recognition model based on YOLOv8n, intelligent response to teacher gestures, such as character erasing, saving and revocation operation, is realized; through a synchronous memory model constructed by a knowledge graph technology, synchronous memory storage of characters and audios is realized, and a synchronous intelligent note is generated; a complete synchronous intelligent note data set can be formed, and an intelligent auxiliary tool with high efficiency and high interactivity is provided for teaching.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent blackboards, and in particular to an intelligent blackboard synchronous memory display method and system. Background Art

[0002] In the modern educational environment, smart blackboards have become an important tool for interaction between teachers and students. Not only do they allow teachers and students to write on the screen using their fingers or a stylus, they also have a handwriting recognition function that digitizes the written content for easy storage and sharing. In addition, the multimedia functions of smart blackboards, such as playing videos and displaying pictures, further enhance the effectiveness of classroom teaching. However, with long-term use, traditional smart blackboards as electronic display screens may emit blue light, affecting students' eyesight, and staring at the screen for a long time may also cause eye fatigue. Some multimedia functions may consume students' concentration and reduce learning efficiency.

[0003] Therefore, it is particularly important to develop a new type of smart blackboard synchronous memory display method. It is necessary to retain the core functions of the smart blackboard, the intelligent recording and active analysis functions of the written content, and the ability to save the written content in real time and use data analysis technology to intelligently summarize class notes; at the same time, the display parameters can be adjusted to reduce blue light radiation and optimize screen brightness and contrast, thereby achieving eye protection effects, which helps to maximize the effectiveness of the smart blackboard while controlling costs. Summary of the invention

[0004] The present invention aims to provide an intelligent blackboard synchronous memory display method and system.

[0005] A method for synchronous memory display of an intelligent blackboard comprises the following steps: Step S1: Obtain the display brightness and ambient light brightness of the location of the smart blackboard to obtain the real-time location ambient light; obtain the distance between the location of the smart blackboard and the visual center of the target teacher and all students to obtain the real-time display first distance and the real-time display second distance; analyze based on the real-time location ambient light, the real-time display first distance, the real-time display second distance and the smart blackboard display adjustment model to obtain the real-time adjustment parameters of the smart blackboard; the smart blackboard display adjustment model includes a region segmentation layer, a viewing distance region matching layer, a light adjustment layer and a display adjustment output layer, which is fused and constructed based on the CNN model and the MLP model to adjust the display parameters of the smart blackboard in real time; Step S2: Real-time monitoring of the target teacher's gestures, including: When the first gesture action is detected, a text erasing operation is performed; when the second gesture action is detected, a text saving operation is performed; when the third gesture action is detected, an undo operation is performed; the text erasing operation, the text saving operation, and the undo operation are all intelligent gesture operations of the smart blackboard, and the intelligent gesture operation recognition of the smart blackboard is performed based on the intelligent blackboard infrared recognition model; the intelligent blackboard infrared recognition model includes an image capture layer, an identity authentication layer, a gesture matching layer, and an operation output layer, and is constructed based on the improved YOLOv8n model to capture the real-time gestures of the target teacher and perform corresponding operations; Step S3: when a text saving operation is received, the current text to be analyzed and the current audio to be analyzed are obtained; the current text to be analyzed and the current audio to be analyzed are input into the intelligent blackboard synchronous memory model for synchronous memory saving to obtain synchronous intelligent notes; the intelligent blackboard synchronous memory model includes a speech recognition layer, a knowledge network construction layer and a synchronous memory output layer, which is constructed based on knowledge graph technology and is used for synchronous memory summary using the intelligent blackboard; Step S4: In a target course, steps S1, S2 and S3 are repeated simultaneously until the target course is finished, and a target course synchronous memory display process is completed to obtain a synchronous intelligent note data set; and then subsequent operations are performed on the synchronous intelligent note data set.

[0006] As a preferred technical solution of the present invention, the intelligent blackboard display adjustment model in step S1 includes a region segmentation layer, a viewing distance region matching layer, a lighting adjustment layer and a display adjustment output layer; The region segmentation layer is used to perform region segmentation operations on the smart blackboard based on the real-time location and ambient lighting to obtain the smart blackboard segmentation region B n , n=1, 2, ..., N; N is the total number of regions that the smart blackboard is segmented into; combined with the real-time location ambient lighting and the smart blackboard segmentation area B n Constructing intelligent blackboard area lighting pair G n , G n =G(B n ), smart blackboard area lighting effect on G n Indicates the intelligent blackboard segmentation area B n Average ambient light The view range matching layer is used to match the illumination of the smart blackboard area to G n The first distance and the second distance are displayed in real time for feature extraction, and the illumination feature T of the intelligent blackboard area to be adjusted is obtained. n ; The view range region matching layer consists of M convolutional layers J m , M pooling layers C m and fully connected layers, m=1, 2, ..., M; the smart blackboard area illumination is G nThe real-time display of the first distance and the real-time display of the second distance are extracted through the viewing area matching layer to obtain the illumination feature T of the smart blackboard area to be adjusted. n ; The illumination adjustment layer is constructed based on the MLP model. The LSTM layer and the self-attention layer are added to the middle layer of the MLP model for fusion connection construction. The illumination adjustment layer is used to adjust the illumination features T of the smart blackboard area. n Perform nonlinear mapping to obtain real-time adjustment parameters of the intelligent blackboard; The display adjustment output layer is used to output the real-time adjustment parameters of the intelligent blackboard.

[0007] As a preferred technical solution of the present invention, the specific steps of constructing the illumination adjustment layer include: Construct K simulated illumination adjustment layer hyperparameter individuals H k , k=1, 2, ..., K; each simulated illumination adjustment layer hyperparameter individual H k Contains a set of hyperparameters for constructing a lighting adjustment layer; K simulated lighting adjustment layer hyperparameter individuals H k Combine to obtain the simulated illumination adjustment hyperparameter iterative population; set the maximum number of iterations; Construct a light adjustment training set to evaluate fitness, based on the simulated light adjustment layer hyperparameter individual H k Perform simulation construction to obtain a simulated lighting adjustment layer; use the lighting adjustment training set and the simulated lighting adjustment layer to perform simulation adjustment to obtain the adjustment accuracy; use the adjustment accuracy as the hyperparameter individual H of the simulated lighting adjustment layer k The fitness S k ; When performing population iteration, set the population iteration hybridization factor Q(e); Q(e) = [(DD min ) / (D max -D min )]*(1-e / E); Q'(e)=clip(Q(e),Q min , Q max ); where E is the maximum number of iterations, e is the current number of iterations, D is the quantitative population diversity, and D max Indicates the maximum population diversity, D min represents the minimum population diversity, Q'(e) is the clipping factor after preprocessing the population iterative hybridization factor Q(e), Q max To set the maximum value of the population iterative hybridization factor, Q min To set the minimum value of population iterative hybridization factor; When the clipping factor Q'(e)>(Q min +Q max) / 2, the adaptive crossover strategy is implemented to update the simulated illumination adjustment hyperparameter iterative population; when the clipping factor Q'(e)<(Q min +Q max ) / 2, the adaptive local search strategy is used to update the simulated illumination adjustment hyperparameter iterative population; When the maximum number of iterations is reached, the simulated lighting adjustment layer hyperparameter individual corresponding to the maximum output fitness is the optimal simulated lighting adjustment layer hyperparameter individual; the lighting adjustment layer is constructed using the optimal simulated lighting adjustment layer hyperparameter individual.

[0008] As a preferred technical solution of the present invention, the intelligent blackboard infrared recognition model in step S2 includes an image capture layer, an identity authentication layer, a gesture matching layer and an operation output layer; The image capture layer is used to capture gestures on the smart blackboard using infrared technology to obtain gesture images to be recognized; The identity authentication layer is used to perform identity authentication based on the gesture image to be recognized and obtain the identity authentication result; if the identity authentication result is passed, the gesture image to be recognized is input into the gesture matching layer for subsequent operations; otherwise, the intelligent inner panel infrared recognition model is exited and no operation is performed; The identity authentication layer is constructed based on the CNN model, and the authenticated gesture image training set is collected; the CNN model is trained using the authenticated gesture image training set to obtain the trained identity authentication layer; The gesture matching layer is used to match gestures according to the gesture image to be recognized and obtain gesture recognition matching results; the gesture matching layer is constructed based on the improved YOLOv8n model; The operation output layer is used to match the smart blackboard smart gesture operation based on the gesture recognition matching results, and output the corresponding smart blackboard smart gesture operation.

[0009] As a preferred technical solution of the present invention, the specific steps of training the gesture matching layer include: The gesture matching layer is constructed using the improved YOLOv8n model, and a four-layer network structure is constructed, namely the input layer, the backbone layer, the neck layer, and the detection head layer; In the backbone layer, the features of the gesture image to be recognized are extracted to obtain the features of the gesture image to be recognized; based on the features of the gesture image to be recognized and the adaptive sampling point offset function, the size of the standard convolution kernel in the backbone layer is adjusted in real time to obtain an adaptively adjusted convolution kernel; the adaptive sampling point offset function is adaptively adjusted based on the features of the gesture image to be recognized; after the adaptive adjustment of the convolution kernel, the high-level features of the gesture image to be recognized are output; The high-level features of the gesture image to be recognized are input into the neck layer and the detection head layer for feature recognition to obtain the gesture recognition matching results; Collect several groups of recognized gesture image samples with target value labels; the target value labels are matching gesture recognition results; combine several groups of recognized gesture image samples with target value labels to obtain a gesture image sample training set; input the gesture image sample training set into the smart blackboard infrared recognition model to perform model training on the gesture matching layer to obtain an initial gesture matching layer; if the initial gesture matching layer passes the model evaluation, the initial gesture matching layer is used as the gesture matching layer in the smart blackboard infrared recognition model; otherwise, continue model training using the gesture image sample training set.

[0010] As a preferred technical solution of the present invention, the intelligent black synchronous memory model in step S3 includes a speech recognition layer, a knowledge network construction layer and a synchronous memory output layer; The speech recognition layer is used to convert the current audio to be analyzed into the current audio text using speech recognition technology; The knowledge network construction layer is used to build based on the knowledge graph technology. The knowledge network construction layer includes the knowledge extraction layer, the knowledge fusion layer and the knowledge connection layer, which specifically include: The knowledge extraction layer is used to extract knowledge from the current audio text and the current text to be analyzed to obtain several related knowledge triples; The knowledge fusion layer is used to perform data alignment operations on several related knowledge triples. First, the domain features of several related knowledge triples are analyzed to obtain several related knowledge core triples and corresponding several related knowledge associated triples. Then, the features of several related knowledge associated triples are fused and connected with the corresponding related knowledge core triples to obtain several related knowledge data groups. The knowledge connection layer is used to integrate several related knowledge data groups using the trained knowledge connection model to obtain synchronized intelligent notes; The synchronous memory output layer is used to output synchronous smart notes.

[0011] As a preferred technical solution of the present invention, the specific steps of constructing the knowledge connection layer in the knowledge network construction layer include: Build the initial knowledge connection layer based on the AI ​​language big model; Collect several groups of knowledge connection training samples; each group of knowledge connection training samples contains a verified knowledge data group and a corresponding grammatical connection language; combine several groups of knowledge connection training samples to obtain a knowledge connection training set; input the knowledge connection training set into the initial knowledge connection layer for training to obtain a knowledge connection layer to be evaluated; perform model evaluation on the knowledge connection layer to be evaluated to obtain a model evaluation result of the knowledge connection layer to be evaluated; if the model evaluation result of the knowledge connection layer to be evaluated is passed, the knowledge connection layer to be evaluated is used as the knowledge connection layer in the knowledge network construction layer; otherwise, continue model training using the knowledge connection training set.

[0012] An intelligent blackboard synchronous memory display system, comprising: The intelligent blackboard display adjustment module includes a data acquisition unit and a display adjustment unit; the data acquisition unit is used to acquire the display brightness and ambient light brightness of the position of the intelligent blackboard to obtain the real-time position ambient light; the distance between the position of the intelligent blackboard and the visual center of the target teacher and all students is acquired to obtain the real-time display first distance and the real-time display second distance; the display adjustment unit is used to analyze based on the real-time position ambient light, the real-time display first distance, the real-time display second distance and the intelligent blackboard display adjustment model to obtain the real-time adjustment parameters of the intelligent blackboard; the intelligent blackboard display adjustment model includes a region segmentation layer, a viewing distance region matching layer, a light adjustment layer and a display adjustment output layer, which is constructed based on the fusion of the CNN model and the MLP model and is used to adjust the display parameters of the intelligent blackboard in real time; The intelligent blackboard synchronous memory module includes a gesture monitoring unit and a synchronous memory unit; the gesture monitoring unit is used to monitor the gestures of the target teacher in real time, specifically including: when the first gesture is detected, a text erasing operation is taken; when the second gesture is detected, a text saving operation is taken; when the third gesture is detected, a revocation operation is taken; the text erasing operation, the text saving operation, and the revocation operation are all intelligent gesture operations of the intelligent blackboard, and the intelligent gesture operation recognition of the intelligent blackboard is performed based on the intelligent blackboard infrared recognition model; the intelligent blackboard infrared recognition model includes an image capture layer, an identity authentication layer, a gesture matching layer, and an operation output layer, which is constructed based on the improved YOLOv8n model and is used to capture the real-time gestures of the target teacher and perform corresponding operations; the synchronous memory unit is used to obtain the current text to be analyzed and the current audio to be analyzed when a text saving operation is received; the current text to be analyzed and the current audio to be analyzed are input into the intelligent blackboard synchronous memory model for synchronous memory preservation to obtain synchronous intelligent notes; the intelligent blackboard synchronous memory model includes a speech recognition layer, a knowledge network construction layer, and a synchronous memory output layer, which is constructed based on the knowledge graph technology and is used to perform synchronous memory summary using the intelligent blackboard; The intelligent blackboard startup control module includes a course startup unit; the course startup unit is used to repeat steps S1, S2 and S3 simultaneously in a target course until the target course ends, complete a target course synchronization memory display process, and obtain a synchronized intelligent note data set; then perform subsequent operations on the synchronized intelligent note data set.

[0013] The present invention has the following advantages: 1. The present invention dynamically adjusts display parameters to optimize the visual experience by real-time analysis of ambient lighting, viewing distance and teacher gestures. The display adjustment model constructed by the fusion of CNN and MLP models and the gesture recognition model based on YOLOv8n are used to realize intelligent response to teacher gestures, such as text erasing, saving and undoing operations. In addition, the synchronous memory model constructed by knowledge graph technology realizes the synchronous memory preservation of text and audio, and generates synchronous intelligent notes. The present invention is conducive to use throughout the course, and finally forms a complete synchronous intelligent note data set, providing an efficient and interactive intelligent auxiliary tool for teaching.

[0014] 2. The present invention realizes intelligent lighting adjustment for different areas through area segmentation, viewing distance matching and lighting adjustment, optimizes the visual experience, adopts CNN and MLP combined with LSTM and self-attention mechanism, enhances the nonlinear mapping ability of lighting characteristics, and improves the accuracy of adjustment parameters; optimizes the hyperparameters of the lighting adjustment layer by simulating evolutionary algorithm, can adapt to different environments and achieve better display effects; improves the display quality of the smart blackboard under different lighting conditions, enhances teaching interactivity, and provides a more comfortable and efficient teaching environment for teachers and students. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a schematic diagram of the structure of an intelligent blackboard synchronous memory display system used in an embodiment of the present invention.

[0016] Figure 2 The present invention is a flowchart of a smart blackboard synchronous memory display method adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0018] Embodiment 1, a kind of intelligent blackboard synchronous memory display method, see Figure 2 As shown, the following steps are included: Step S1: Obtain the display brightness and ambient light brightness of the location of the smart blackboard to obtain the real-time location ambient light; obtain the distance between the location of the smart blackboard and the visual center of the target teacher and all students to obtain the real-time display first distance and the real-time display second distance; analyze based on the real-time location ambient light, the real-time display first distance, the real-time display second distance and the smart blackboard display adjustment model to obtain the real-time adjustment parameters of the smart blackboard; the smart blackboard display adjustment model includes a region segmentation layer, a viewing distance region matching layer, a light adjustment layer and a display adjustment output layer, which is fused and constructed based on the CNN model and the MLP model to adjust the display parameters of the smart blackboard in real time; The built-in sensor on the smart blackboard is used to obtain the display brightness, ambient light brightness and visual center distance of the location of the smart blackboard; The intelligent blackboard display adjustment model in step S1 includes a region segmentation layer, a viewing distance region matching layer, a lighting adjustment layer and a display adjustment output layer; The region segmentation layer is used to perform region segmentation operations on the smart blackboard based on the real-time location and ambient lighting to obtain the smart blackboard segmentation region B n , n=1, 2, ..., N; N is the total number of regions that the smart blackboard is segmented into; combined with the real-time location ambient lighting and the smart blackboard segmentation area B n Constructing intelligent blackboard area lighting pair G n , G n =G(B n ), smart blackboard area lighting effect on G n Indicates the intelligent blackboard segmentation area B n Average ambient light The view range matching layer is used to match the illumination of the smart blackboard area to G n The first distance and the second distance are displayed in real time for feature extraction, and the illumination feature T of the intelligent blackboard area to be adjusted is obtained. n ; The view range region matching layer consists of M convolutional layers J m , M pooling layers C m and fully connected layers, m=1, 2, ..., M; the smart blackboard area illumination is G n The real-time display of the first distance and the real-time display of the second distance are extracted through the viewing area matching layer to obtain the illumination feature T of the smart blackboard area to be adjusted. n ; The illumination adjustment layer is constructed based on the MLP model. The LSTM layer and the self-attention layer are added to the middle layer of the MLP model for fusion connection construction. The illumination adjustment layer is used to adjust the illumination features T of the smart blackboard area. n Perform nonlinear mapping to obtain real-time adjustment parameters of the intelligent blackboard; The display adjustment output layer is used to output the real-time adjustment parameters of the intelligent blackboard; Through regional segmentation, the model can identify the lighting differences in different areas and make more precise adjustments, which helps to improve the visibility and clarity of the content on the blackboard under different lighting conditions. By combining the feature extraction capabilities of the convolutional layer and the pooling layer, and the integration capabilities of the fully connected layer, the viewing distance region matching layer can effectively capture and process the complex lighting features related to the viewing distance, which enables the model to adjust the display effect according to the distance of the observer, thereby improving the user experience. The MLP model provides powerful nonlinear mapping capabilities, while the LSTM layer can capture time series information, and the self-attention layer enhances the model's ability to focus on important features. This combination enables the lighting adjustment layer to dynamically adapt to changing lighting conditions and provide precise adjustment parameters. The display adjustment output layer realizes intelligent optimization of the display effect of the smart blackboard, ensuring optimal readability under different ambient lighting and viewing distance conditions. The specific steps to build a lighting adjustment layer include: Construct K simulated illumination adjustment layer hyperparameter individuals H k , k=1, 2, ..., K; each simulated illumination adjustment layer hyperparameter individual H k Contains a set of hyperparameters for constructing a lighting adjustment layer; K simulated lighting adjustment layer hyperparameter individuals H k Combination, to obtain the simulated illumination adjustment hyperparameter iteration population; set the maximum number of iterations; the maximum number of iterations is set by professional technicians according to actual conditions; Construct a light adjustment training set to evaluate fitness, based on the simulated light adjustment layer hyperparameter individual H k Perform simulation construction to obtain a simulated lighting adjustment layer; use the lighting adjustment training set and the simulated lighting adjustment layer to perform simulation adjustment to obtain the adjustment accuracy; use the adjustment accuracy as the hyperparameter individual H of the simulated lighting adjustment layer k The fitness S k ; When performing population iteration, set the population iteration hybridization factor Q(e); Q(e)=[(DD min ) / (D max -D min )]*(1-e / E); Q'(e)=clip(Q(e),Q min , Q max ); where E is the maximum number of iterations, e is the current number of iterations, D is the quantitative population diversity, and D max Indicates the maximum population diversity, D min represents the minimum population diversity, Q'(e) is the clipping factor after preprocessing the population iterative hybridization factor Q(e), Q max To set the maximum value of the population iterative hybridization factor, Q minTo set the minimum value of the population iteration hybridization factor; the maximum value of population diversity and the minimum value of population diversity are set by professional technicians according to actual conditions; the quantified population diversity can be simulated and quantified by calculating the Euclidean distance and other methods; setting the maximum value of the population iteration hybridization factor and setting the minimum value of the population iteration hybridization factor fix the range of variation of the population iteration hybridization factor Q (e), and it can be assumed that the maximum value of the population iteration hybridization factor is set to 0.9, and the minimum value of the population iteration hybridization factor is set to 0.1; When the clipping factor Q'(e)>(Q min +Q max ) / 2, the adaptive crossover strategy is implemented to update the simulated illumination adjustment hyperparameter iterative population; when the clipping factor Q'(e)<(Q min +Q max ) / 2, the adaptive local search strategy is used to update the simulated illumination adjustment hyperparameter iterative population; When the maximum number of iterations is reached, the simulated illumination adjustment layer hyperparameter individual corresponding to the maximum fitness is output, which is the optimal simulated illumination adjustment layer hyperparameter individual; the illumination adjustment layer is constructed using the optimal simulated illumination adjustment layer hyperparameter individual; By simulating evolutionary strategies to optimize the lighting adjustment layer of the smart blackboard, the adaptability and flexibility of the display system are improved. By constructing multiple hyperparameter individuals and forming an iterative population, the fitness is evaluated in combination with the lighting adjustment training set, ensuring that the model can be effectively adjusted to different environmental lighting conditions. The dynamic adjustment strategy of the population iterative hybridization factor enables the algorithm to gradually converge to the optimal solution while maintaining diversity. This method not only improves the display quality of the smart blackboard in various lighting environments, but also enhances the system's adaptive ability and the user's visual comfort, providing more intelligent and personalized display support for teaching. Step S2: Real-time monitoring of the target teacher's gestures, including: When the first gesture action is detected, a text erasing operation is performed; when the second gesture action is detected, a text saving operation is performed; when the third gesture action is detected, an undo operation is performed; the text erasing operation, the text saving operation, and the undo operation are all intelligent gesture operations of the smart blackboard, and the intelligent gesture operation recognition of the smart blackboard is performed based on the intelligent blackboard infrared recognition model; the intelligent blackboard infrared recognition model includes an image capture layer, an identity authentication layer, a gesture matching layer, and an operation output layer, and is constructed based on the improved YOLOv8n model to capture the real-time gestures of the target teacher and perform corresponding operations; The text erasing operation is to erase the text covered by the gesture based on the first gesture action; the text saving operation is to save all the text on the current smart blackboard, clear all the text on the smart blackboard, and open the next page; the undo operation is to withdraw the previous step; The intelligent blackboard infrared recognition model in step S2 includes an image capture layer, an identity authentication layer, a gesture matching layer and an operation output layer; The image capture layer is used to capture gestures on the smart blackboard using infrared technology to obtain gesture images to be recognized; The identity authentication layer is used to perform identity authentication based on the gesture image to be recognized and obtain the identity authentication result; if the identity authentication result is passed, the gesture image to be recognized is input into the gesture matching layer for subsequent operations; otherwise, the intelligent inner panel infrared recognition model is exited and no operation is performed; The identity authentication layer is constructed based on the CNN model, and the authenticated gesture image training set is collected; the CNN model is trained using the authenticated gesture image training set to obtain the trained identity authentication layer; The gesture matching layer is used to match gestures according to the gesture image to be recognized and obtain gesture recognition matching results; the gesture matching layer is constructed based on the improved YOLOv8n model; The operation output layer is used to match the smart blackboard smart gesture operation based on the gesture recognition matching results and output the corresponding smart blackboard smart gesture operation; The use of infrared technology makes gesture capture unaffected by changes in ambient light, and relies on temperature changes for recognition, which improves the reliability and accuracy of gesture detection. This layer can quickly capture gesture images and provide basic data for subsequent recognition and operation; for example, when the gesture is close to the smart blackboard to a certain distance, the gesture will be recognized; by using the CNN model for identity authentication, it can ensure that only authorized teacher gestures can trigger corresponding operations. This layer enhances security and reliability and prevents students from unauthorized use; using the improved YOLOv8n model, the gesture matching layer can quickly and accurately identify different gestures. The YOLO model is known for its efficient real-time detection capabilities, which ensures the speed and accuracy of gesture recognition and helps to respond to the teacher's gesture instructions immediately; the operation output layer converts the gesture recognition results into specific operation instructions, such as text erase, save, undo, etc., to achieve the intuitiveness and convenience of gesture control. By quickly outputting operation instructions, it can respond to teachers' needs in a timely manner and improve classroom interaction efficiency; The specific steps for training the gesture matching layer include: The gesture matching layer is constructed using the improved YOLOv8n model, and a four-layer network structure is constructed, namely the input layer, the backbone layer, the neck layer, and the detection head layer; In the backbone layer, the features of the gesture image to be recognized are extracted to obtain the features of the gesture image to be recognized; based on the features of the gesture image to be recognized and the adaptive sampling point offset function, the size of the standard convolution kernel in the backbone layer is adjusted in real time to obtain an adaptively adjusted convolution kernel; the adaptive sampling point offset function is adaptively adjusted based on the features of the gesture image to be recognized; after the adaptive adjustment of the convolution kernel, the high-level features of the gesture image to be recognized are output; The high-level features of the gesture image to be recognized are input into the neck layer and the detection head layer for feature recognition to obtain the gesture recognition matching results; Collect several groups of recognized gesture image samples with target value labels; the target value labels are matching gesture recognition results; combine several groups of recognized gesture image samples with target value labels to obtain a gesture image sample training set; input the gesture image sample training set into the smart blackboard infrared recognition model to perform model training on the gesture matching layer to obtain an initial gesture matching layer; if the initial gesture matching layer passes the model evaluation, the initial gesture matching layer is used as the gesture matching layer in the smart blackboard infrared recognition model; otherwise, continue model training using the gesture image sample training set; The gesture matching layer is constructed through the improved YOLOv8n model, which realizes efficient recognition of gesture images; the convolution kernel size is adjusted in real time through the adaptive sampling point offset function, which enhances the model's ability to capture different gesture features and improves the recognition accuracy; the design of the four-layer network structure, including the input layer, the backbone layer, the neck layer and the detection head layer, ensures smooth processing from feature extraction to recognition and matching; in addition, by training with gesture image samples with target value labels, the model can learn and adapt to specific gesture recognition tasks. This method improves the flexibility and accuracy of the smart blackboard in gesture recognition, enhances the user interaction experience, and provides an intuitive and natural interaction method for teaching and control; Step S3: when a text saving operation is received, the current text to be analyzed and the current audio to be analyzed are obtained; the current text to be analyzed and the current audio to be analyzed are input into the intelligent blackboard synchronous memory model for synchronous memory saving to obtain synchronous intelligent notes; the intelligent blackboard synchronous memory model includes a speech recognition layer, a knowledge network construction layer and a synchronous memory output layer, which is constructed based on knowledge graph technology and is used for synchronous memory summary using the intelligent blackboard; The intelligent black synchronous memory model in step S3 includes a speech recognition layer, a knowledge network construction layer and a synchronous memory output layer; The speech recognition layer is used to convert the current audio to be analyzed into the current audio text using speech recognition technology; The knowledge network construction layer is used to build based on the knowledge graph technology. The knowledge network construction layer includes the knowledge extraction layer, the knowledge fusion layer and the knowledge connection layer, which specifically include: The knowledge extraction layer is used to extract knowledge from the current audio text and the current text to be analyzed to obtain several related knowledge triples; The knowledge fusion layer is used to perform data alignment operations on several related knowledge triples. First, the domain features of several related knowledge triples are analyzed to obtain several related knowledge core triples and corresponding several related knowledge associated triples. Then, the features of several related knowledge associated triples are fused and connected with the corresponding related knowledge core triples to obtain several related knowledge data groups. The knowledge connection layer is used to integrate several related knowledge data groups using the trained knowledge connection model to obtain synchronized intelligent notes; The synchronous memory output layer is used to output synchronous smart notes; Through speech recognition technology, the speech recognition layer can quickly and accurately convert audio content into text, so that audio information can be combined with written content for further analysis and processing, which improves the accessibility and processing efficiency of information; by extracting knowledge triples from the current audio text and the text to be analyzed, it can identify and extract key knowledge points and relationships to form a preliminary knowledge structure; by aligning data and fusing features of knowledge triples, it can eliminate redundant information, integrate knowledge from different sources, and form a consistent knowledge representation, which helps to improve the integrity and accuracy of the knowledge network; using the trained large model for knowledge integration can further optimize the structure of the knowledge network and generate synchronous intelligent notes with logic and coherence. This process ensures that the generated notes not only contain important information, but also present it in an easy-to-understand way; converting complex knowledge networks into user-readable notes to provide users with a clear and structured summary, which enables users to quickly review and understand classroom content and improve learning efficiency; The specific steps of constructing the knowledge connection layer in the knowledge network construction layer include: Build the initial knowledge connection layer based on the AI ​​language big model; Collect several groups of knowledge connection training samples; each group of knowledge connection training samples contains a verified knowledge data group and a corresponding grammatical connection language; combine several groups of knowledge connection training samples to obtain a knowledge connection training set; input the knowledge connection training set into the initial knowledge connection layer for training to obtain a knowledge connection layer to be evaluated; perform model evaluation on the knowledge connection layer to be evaluated to obtain a model evaluation result of the knowledge connection layer to be evaluated; if the model evaluation result of the knowledge connection layer to be evaluated is passed, the knowledge connection layer to be evaluated is used as the knowledge connection layer in the knowledge network construction layer; otherwise, continue model training using the knowledge connection training set; By using the AI ​​language big model to build the initial knowledge connection layer and combining it with verified knowledge data and grammatical connection language for training, the accuracy and practicality of the knowledge network construction layer are improved. This method can systematically integrate and connect different knowledge points to form a structured knowledge network. Through model evaluation and iterative training, the quality of the knowledge connection layer is ensured, so that it can more effectively support the synchronous memory and knowledge summary functions of the smart blackboard. This method not only improves the intelligence level of intelligent teaching tools, but also enhances its application value in the field of education, providing students and teachers with richer and more dynamic learning resources. Step S4: in a target course, repeat steps S1, S2 and S3 simultaneously until the target course is finished, completing a target course synchronous memory display process, and obtaining a synchronized intelligent note data set; and then performing subsequent operations on the synchronized intelligent note data set; In actual use, when using the smart blackboard for classroom assignments, efficient text recording and real-time transmission can be achieved. Classroom content, including text, audio and visual information, can be continuously captured and synchronized, thereby generating a detailed synchronized smart note data set, reducing the need for manual recording and improving the efficiency and accuracy of recording; it can capture and transmit information in real time to ensure that all participants have instant access to the latest content, enhancing the timeliness of information; the generated synchronized smart note data set can be used for a variety of subsequent operations, such as content review, knowledge management, data analysis, etc., increasing the availability and flexibility of the data; in addition to being used for classroom assignments, the usage scenarios can also be expanded to real-time meetings, etc., with strong scalability.

[0019] Embodiment 2, a smart blackboard synchronous memory display system, see Figure 1 As shown, including: The intelligent blackboard display adjustment module includes a data acquisition unit and a display adjustment unit; the data acquisition unit is used to acquire the display brightness and ambient light brightness of the position of the intelligent blackboard to obtain the real-time position ambient light; the distance between the position of the intelligent blackboard and the visual center of the target teacher and all students is acquired to obtain the real-time display first distance and the real-time display second distance; the display adjustment unit is used to analyze based on the real-time position ambient light, the real-time display first distance, the real-time display second distance and the intelligent blackboard display adjustment model to obtain the real-time adjustment parameters of the intelligent blackboard; the intelligent blackboard display adjustment model includes a region segmentation layer, a viewing distance region matching layer, a light adjustment layer and a display adjustment output layer, which is constructed based on the fusion of the CNN model and the MLP model and is used to adjust the display parameters of the intelligent blackboard in real time; The intelligent blackboard synchronous memory module includes a gesture monitoring unit and a synchronous memory unit; the gesture monitoring unit is used to monitor the gestures of the target teacher in real time, specifically including: when the first gesture is detected, a text erasing operation is taken; when the second gesture is detected, a text saving operation is taken; when the third gesture is detected, a revocation operation is taken; the text erasing operation, the text saving operation, and the revocation operation are all intelligent gesture operations of the intelligent blackboard, and the intelligent gesture operation recognition of the intelligent blackboard is performed based on the intelligent blackboard infrared recognition model; the intelligent blackboard infrared recognition model includes an image capture layer, an identity authentication layer, a gesture matching layer, and an operation output layer, which is constructed based on the improved YOLOv8n model and is used to capture the real-time gestures of the target teacher and perform corresponding operations; the synchronous memory unit is used to obtain the current text to be analyzed and the current audio to be analyzed when a text saving operation is received; the current text to be analyzed and the current audio to be analyzed are input into the intelligent blackboard synchronous memory model for synchronous memory preservation to obtain synchronous intelligent notes; the intelligent blackboard synchronous memory model includes a speech recognition layer, a knowledge network construction layer, and a synchronous memory output layer, which is constructed based on the knowledge graph technology and is used to perform synchronous memory summary using the intelligent blackboard; The intelligent blackboard startup control module includes a course startup unit; the course startup unit is used to repeat steps S1, S2 and S3 simultaneously in a target course until the target course ends, complete a target course synchronization memory display process, and obtain a synchronized intelligent note data set; then perform subsequent operations on the synchronized intelligent note data set.

[0020] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A method for synchronous memory display of an intelligent blackboard, characterized in that: The following steps are involved: Step S1: Obtain the display brightness and ambient light brightness of the location of the smart blackboard to obtain the real-time location ambient light; obtain the distance between the location of the smart blackboard and the visual center of the target teacher and all students to obtain the real-time display first distance and the real-time display second distance; analyze based on the real-time location ambient light, the real-time display first distance, the real-time display second distance and the smart blackboard display adjustment model to obtain the real-time adjustment parameters of the smart blackboard; the smart blackboard display adjustment model includes a region segmentation layer, a viewing distance region matching layer, a light adjustment layer and a display adjustment output layer, which is fused and constructed based on the CNN model and the MLP model to adjust the display parameters of the smart blackboard in real time; Step S2: Real-time monitoring of the target teacher's gestures, including: When the first gesture action is detected, a text erasing operation is performed; when the second gesture action is detected, a text saving operation is performed; when the third gesture action is detected, an undo operation is performed; the text erasing operation, the text saving operation, and the undo operation are all intelligent gesture operations of the smart blackboard, and the intelligent gesture operation recognition of the smart blackboard is performed based on the intelligent blackboard infrared recognition model; the intelligent blackboard infrared recognition model includes an image capture layer, an identity authentication layer, a gesture matching layer, and an operation output layer, and is constructed based on the improved YOLOv8n model to capture the real-time gestures of the target teacher and perform corresponding operations; Step S3: when a text saving operation is received, the current text to be analyzed and the current audio to be analyzed are obtained; the current text to be analyzed and the current audio to be analyzed are input into the intelligent blackboard synchronous memory model for synchronous memory saving to obtain synchronous intelligent notes; the intelligent blackboard synchronous memory model includes a speech recognition layer, a knowledge network construction layer and a synchronous memory output layer, which is constructed based on knowledge graph technology and is used for synchronous memory summary using the intelligent blackboard; Step S4: In a target course, steps S1, S2 and S3 are repeated simultaneously until the target course is finished, and a target course synchronous memory display process is completed to obtain a synchronous intelligent note data set; and then subsequent operations are performed on the synchronous intelligent note data set.

2. A smart blackboard synchronous memory display method according to claim 1, characterized in that: The intelligent blackboard display adjustment model in step S1 includes a region segmentation layer, a viewing distance region matching layer, a lighting adjustment layer and a display adjustment output layer; The region segmentation layer is used to perform region segmentation operations on the smart blackboard based on the real-time location and ambient lighting to obtain the smart blackboard segmentation region B n , n=1,2,…,N; N is the total number of regions that the smart blackboard is segmented into; combined with the real-time location ambient lighting and the smart blackboard segmentation area B n Constructing intelligent blackboard area lighting pair G n , G n =G(B n ), smart blackboard area lighting effect on G n Indicates the intelligent blackboard segmentation area B n Average ambient light The view range matching layer is used to match the illumination of the smart blackboard area to G n The first distance and the second distance are displayed in real time for feature extraction, and the illumination feature T of the intelligent blackboard area to be adjusted is obtained. n ; The view range region matching layer consists of M convolutional layers J m , M pooling layers C m And fully connected layer, m=1,2,…,M; The smart blackboard area lighting is aligned with G n The real-time display of the first distance and the real-time display of the second distance are extracted through the viewing area matching layer to obtain the illumination feature T of the smart blackboard area to be adjusted. n ; The illumination adjustment layer is constructed based on the MLP model. The LSTM layer and the self-attention layer are added to the middle layer of the MLP model for fusion connection construction. The lighting adjustment layer is used to adjust the lighting characteristics of the smart blackboard area. n Perform nonlinear mapping to obtain real-time adjustment parameters of the intelligent blackboard; The display adjustment output layer is used to output the real-time adjustment parameters of the intelligent blackboard.

3. A kind of intelligent blackboard synchronous memory display method according to claim 2, it is characterized in that, The specific steps to build a lighting adjustment layer include: Construct K simulated illumination adjustment layer hyperparameter individuals H k , k=1, 2, ..., K; each simulated illumination adjustment layer hyperparameter individual H k Contains a set of hyperparameters for constructing a lighting adjustment layer; K simulated lighting adjustment layer hyperparameter individuals H k Combine to obtain the simulated illumination adjustment hyperparameter iterative population; set the maximum number of iterations; Construct a light adjustment training set to evaluate fitness, based on the simulated light adjustment layer hyperparameter individual H k Perform simulation construction to obtain a simulated lighting adjustment layer; use the lighting adjustment training set and the simulated lighting adjustment layer to perform simulation adjustment to obtain the adjustment accuracy; use the adjustment accuracy as the hyperparameter individual H of the simulated lighting adjustment layer k The fitness S k ; When performing population iteration, set the population iteration hybridization factor Q(e); Q(e)=[(DD min ) / (D max -D min )]*(1-e / E); Q'(e)=clip(Q(e),Q min , Q max ); where E is the maximum number of iterations, e is the current number of iterations, D is the quantitative population diversity, and D max Indicates the maximum population diversity, D min represents the minimum population diversity, Q'(e) is the clipping factor after preprocessing the population iterative hybridization factor Q(e), Q max To set the maximum value of the population iterative hybridization factor, Q min To set the minimum value of population iterative hybridization factor; When the clipping factor Q'(e)>(Q min +Q max ) / 2, the adaptive crossover strategy is implemented to update the simulated illumination adjustment hyperparameter iterative population; when the clipping factor Q'(e)<(Q min +Q max ) / 2, the adaptive local search strategy is used to update the simulated illumination adjustment hyperparameter iterative population; When the maximum number of iterations is reached, the simulated lighting adjustment layer hyperparameter individual corresponding to the maximum output fitness is the optimal simulated lighting adjustment layer hyperparameter individual; the lighting adjustment layer is constructed using the optimal simulated lighting adjustment layer hyperparameter individual.

4. A smart blackboard synchronous memory display method according to claim 3, characterized in that: The intelligent blackboard infrared recognition model in step S2 includes an image capture layer, an identity authentication layer, a gesture matching layer and an operation output layer; The image capture layer is used to capture gestures on the smart blackboard using infrared technology to obtain gesture images to be recognized; The identity authentication layer is used to perform identity authentication based on the gesture image to be recognized and obtain the identity authentication result; if the identity authentication result is passed, the gesture image to be recognized is input into the gesture matching layer for subsequent operations; otherwise, the intelligent inner panel infrared recognition model is exited and no operation is performed; The identity authentication layer is built based on the CNN model and collects a training set of authenticated gesture images; Use the authenticated gesture image training set to train the CNN model to obtain a trained identity authentication layer; The gesture matching layer is used to perform gesture matching based on the gesture image to be recognized and obtain the gesture recognition matching result; The gesture matching layer is built based on the improved YOLOv8n model; The operation output layer is used to match the smart blackboard smart gesture operation based on the gesture recognition matching results, and output the corresponding smart blackboard smart gesture operation.

5. A smart blackboard synchronous memory display method according to claim 4, characterized in that: The specific steps for training the gesture matching layer include: The gesture matching layer is constructed using the improved YOLOv8n model, and a four-layer network structure is constructed, namely the input layer, backbone layer, neck layer, and detection head layer; In the backbone layer, the features of the gesture image to be recognized are extracted to obtain the features of the gesture image to be recognized; based on the features of the gesture image to be recognized and the adaptive sampling point offset function, the size of the standard convolution kernel in the backbone layer is adjusted in real time to obtain an adaptively adjusted convolution kernel; the adaptive sampling point offset function is adaptively adjusted based on the features of the gesture image to be recognized; after the adaptive adjustment of the convolution kernel, the high-level features of the gesture image to be recognized are output; The high-level features of the gesture image to be recognized are input into the neck layer and the detection head layer for feature recognition to obtain the gesture recognition matching results; Collect several groups of recognized gesture image samples with target value labels; the target value labels are matching gesture recognition results; combine several groups of recognized gesture image samples with target value labels to obtain a gesture image sample training set; input the gesture image sample training set into the smart blackboard infrared recognition model to perform model training on the gesture matching layer to obtain an initial gesture matching layer; if the initial gesture matching layer passes the model evaluation, the initial gesture matching layer is used as the gesture matching layer in the smart blackboard infrared recognition model; otherwise, continue model training using the gesture image sample training set.

6. A smart blackboard synchronous memory display method according to claim 5, characterized in that: The intelligent black synchronous memory model in step S3 includes a speech recognition layer, a knowledge network construction layer and a synchronous memory output layer; The speech recognition layer is used to convert the current audio to be analyzed into the current audio text using speech recognition technology; The knowledge network construction layer is used to build based on the knowledge graph technology. The knowledge network construction layer includes the knowledge extraction layer, the knowledge fusion layer and the knowledge connection layer, which specifically include: The knowledge extraction layer is used to extract knowledge from the current audio text and the current text to be analyzed to obtain several related knowledge triples; The knowledge fusion layer is used to perform data alignment operations on several related knowledge triples. First, the domain features of several related knowledge triples are analyzed to obtain several related knowledge core triples and corresponding several related knowledge associated triples. Then, the features of several related knowledge associated triples are fused and connected with the corresponding related knowledge core triples to obtain several related knowledge data groups. The knowledge connection layer is used to integrate several related knowledge data groups using the trained knowledge connection model to obtain synchronized intelligent notes; The synchronous memory output layer is used to output synchronous smart notes.

7. A smart blackboard synchronous memory display method according to claim 6, characterized in that: The specific steps of constructing the knowledge connection layer in the knowledge network construction layer include: Build the initial knowledge connection layer based on the AI ​​language big model; Collect several groups of knowledge connection training samples; each group of knowledge connection training samples contains a verified knowledge data group and a corresponding grammatical connection language; combine several groups of knowledge connection training samples to obtain a knowledge connection training set; input the knowledge connection training set into the initial knowledge connection layer for training to obtain a knowledge connection layer to be evaluated; perform model evaluation on the knowledge connection layer to be evaluated to obtain a model evaluation result of the knowledge connection layer to be evaluated; if the model evaluation result of the knowledge connection layer to be evaluated is passed, the knowledge connection layer to be evaluated is used as the knowledge connection layer in the knowledge network construction layer; otherwise, continue model training using the knowledge connection training set.

8. An intelligent blackboard synchronous memory display system, characterized in that: The system applies a smart blackboard synchronous memory display method according to any one of claims 1 to 7, comprising: The intelligent blackboard display adjustment module includes a data acquisition unit and a display adjustment unit; the data acquisition unit is used to acquire the display brightness and ambient light brightness of the position of the intelligent blackboard to obtain the real-time position ambient light; the distance between the position of the intelligent blackboard and the visual center of the target teacher and all students is acquired to obtain the real-time display first distance and the real-time display second distance; the display adjustment unit is used to analyze based on the real-time position ambient light, the real-time display first distance, the real-time display second distance and the intelligent blackboard display adjustment model to obtain the real-time adjustment parameters of the intelligent blackboard; the intelligent blackboard display adjustment model includes a region segmentation layer, a viewing distance region matching layer, a light adjustment layer and a display adjustment output layer, which is constructed based on the fusion of the CNN model and the MLP model and is used to adjust the display parameters of the intelligent blackboard in real time; The intelligent blackboard synchronous memory module includes a gesture monitoring unit and a synchronous memory unit; the gesture monitoring unit is used to monitor the gestures of the target teacher in real time, specifically including: when the first gesture is detected, a text erasing operation is taken; when the second gesture is detected, a text saving operation is taken; when the third gesture is detected, a revocation operation is taken; the text erasing operation, the text saving operation, and the revocation operation are all intelligent gesture operations of the intelligent blackboard, and the intelligent gesture operation recognition of the intelligent blackboard is performed based on the intelligent blackboard infrared recognition model; the intelligent blackboard infrared recognition model includes an image capture layer, an identity authentication layer, a gesture matching layer, and an operation output layer, which is constructed based on the improved YOLOv8n model and is used to capture the real-time gestures of the target teacher and perform corresponding operations; the synchronous memory unit is used to obtain the current text to be analyzed and the current audio to be analyzed when a text saving operation is received; the current text to be analyzed and the current audio to be analyzed are input into the intelligent blackboard synchronous memory model for synchronous memory preservation to obtain synchronous intelligent notes; the intelligent blackboard synchronous memory model includes a speech recognition layer, a knowledge network construction layer, and a synchronous memory output layer, which is constructed based on the knowledge graph technology and is used to perform synchronous memory summary using the intelligent blackboard; The intelligent blackboard startup control module includes a course startup unit; the course startup unit is used to repeat steps S1, S2 and S3 simultaneously in a target course until the target course ends, complete a target course synchronization memory display process, and obtain a synchronized intelligent note data set; then perform subsequent operations on the synchronized intelligent note data set.

Images