Teaching display system based on AR interaction technology

Abstract

The invention discloses a teaching display system based on an AR interaction technology, which relates to the technical field of education systems and comprises a user terminal equipment module, a teaching resource module, an AR content generation module, an interaction control module, a network communication module, a multi-user AR interaction module and a learning recording and evaluation module. And the user terminal equipment module is used for managing various users and equipment. By optimizing the 3D model to adapt to the terminal device, the AR technology is comprehensively fused into each link of teaching display, the rendering efficiency is improved, accurate reality scene fusion is matched, the reality sense of virtual-real combination is enhanced, diversified teaching requirements and different device display are met, multiple interaction modes are supported, the user operation approach is widened, and the user experience is improved. According to the method, the operation instruction is accurately recognized and processed, the result is fed back in time, the participation sense and learning enthusiasm of the user are enhanced, and compared with a traditional teaching mode, the immersive and high-interactivity learning experience is provided for students, and the teaching form is greatly innovated.

Description

Technical Field

[0001] The present invention relates to the technical field of educational systems, and specifically provides a teaching display system based on AR interaction technology. Background Art

[0002] With the rapid development of information technology, the education field is constantly innovating and changing to meet the needs of the times and the learning characteristics of students. Traditional teaching display methods, such as blackboard writing and slide presentations, mainly present teaching content in a two-dimensional plane. Students are often in a passive state of receiving knowledge, lacking sufficient interactivity and sense of participation. This teaching method restricts the development of students' learning interests and creativity to a certain extent and is difficult to meet the current diversified and personalized educational needs. With the continuous maturity and development of augmented reality (AR) technology, applying AR technology in teaching can transform abstract knowledge into intuitive and vivid three-dimensional models or dynamic scenes, enabling students to more intuitively observe and understand learning content.

[0003] However, in the prior art, existing AR teaching resources are relatively scarce and of uneven quality, and it is easy to have poor adaptability problems for different terminal devices, resulting in situations such as frame drops, blurred displays, or even inability to run properly. Moreover, in AR teaching displays, problems such as inaccurate spatial positioning and inability to synchronize multi-person interactions in real time are also prone to occur, making it difficult to accurately integrate virtual content with the real scene, thus affecting students' learning experience and teaching effects. Summary of the Invention

[0004] The purpose of the present invention is to provide a teaching display system based on AR interaction technology to solve the problems in the above background art, namely, the relatively scarce and uneven-quality existing AR teaching resources, the poor adaptability problems for different terminal devices, resulting in frame drops, blurred displays, or even inability to run properly, and in AR teaching displays, problems such as inaccurate spatial positioning and inability to synchronize multi-person interactions in real time, making it difficult to accurately integrate virtual content with the real scene, thus affecting students' learning experience and teaching effects.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A teaching display system based on AR interaction technology, including a user terminal device module, a teaching resource module, an AR content generation module, an interaction control module, a network communication module, a multi-person AR interaction module, and a learning record and evaluation module; The user terminal device module is used to manage various users and devices, including user registration, login, permission setting, and access management of multi-terminal devices; The teaching resource module is used for teachers to upload teaching resources, classify, store, retrieve, and update the teaching resources, provide functions of automatic annotation and classification description of resources, statistically and update the situation of teaching resources in real time, and at the same time support teachers to create, edit, and manage AR teaching courses; The AR content generation module is used to process and process various materials in the teaching resource module, transform the teaching resources into AR content by using AR technology, and support dynamic interaction functions for 3D models; The interaction control module realizes the interaction between users and virtual content in the teaching display system based on AR interaction technology and supports multiple interaction methods; The network communication module is used to realize the communication between various modules in the system and between the system and external devices; The multi-person AR interaction module is used for multiple users to participate in AR teaching activities at the same time, realize the interaction and collaboration between users, create a multi-person shared AR virtual environment, and enable different users to interact in the same virtual space through spatial synchronous positioning, realizing the sharing and synchronous update of AR content; The learning record and evaluation module is used to record the learning activities and related data of students in the system, evaluate the learning effects of students based on the analysis of learning data, provide teaching feedback for teachers, and also provide learning suggestions and improvement directions for students.

[0006] Preferably, the user terminal device module includes a user registration and login module, a permission management module, and a device configuration management module; The user registration and login module is used to provide a registration entry for new users, support users to input personal basic information, and provide a user login function; The permission management module is used to classify and manage the permissions and responsibilities of different users; The device configuration management module is used to set the parameters of the device according to teaching requirements and device characteristics and allocate device resources.

[0007] Preferably, the teaching resource module includes a resource upload module, a retrieval and recommendation module, and an update and maintenance module; The resource upload module is used for teachers or administrators to have the permission to upload teaching resources to the system, classify and manage the uploaded resources at the same time, review the copyright of the resources, and add metadata to each teaching resource; The retrieval and recommendation module is used to provide a resource retrieval function and intelligently recommend relevant teaching resources for users according to the user's preference data by using algorithms; The update and maintenance module is used to regularly update the teaching resource library, including adding new teaching content, correcting incorrect information, and optimizing the quality of resources.

[0008] Preferably, the AR content generation module includes a 3D model processing module, a real-scene fusion module, a dynamic content generation module, and a content adaptation module; The 3D model processing module is used to import and parse 3D models to ensure that the model formats can be recognized and processed by the system, and to optimize the models; the model optimization processing includes face number simplification and texture compression; For face number simplification, the quadratic error metric algorithm is adopted. For a triangular mesh model, each vertex v has an associated quadratic error matrix. When merging two vertices and into a new vertex the error matrix of the new vertex , the goal is to find the vertex merging operation that minimizes the error. The error . By continuously merging vertices until the required number of faces is reached; For texture compression, the DXT compression algorithm is adopted. The color of each pixel block is represented by two endpoint colors and and a 2-bit index value. The color interpolation formula is: ; ; The real-scene fusion module is used to overlay virtual content at appropriate positions in the real scene according to teaching requirements and the characteristics of the real scene, and to determine the position, angle, and size of the virtual content by using the information provided by the scene recognition and tracking module; The dynamic content generation module is used to generate dynamic AR content in real time according to teaching logic and user interaction operations; The content adaptation module is used to adapt and optimize the generated AR content, and to adjust the resolution, display ratio, and interaction method of the content.

[0009] Preferably, the interaction control module includes an interaction method recognition unit, an interaction logic processing unit, and a feedback presentation unit; The interaction method recognition unit is used to capture user inputs by using various sensors, convert the actual operations of the user into signals that can be processed by the system, analyze the operation intentions, and provide an interaction basis; The interaction logic processing unit is used to receive user instructions from the interaction method recognition unit for conversion, judge and control the operations of the user according to interaction rules and teaching objectives, and manage and coordinate relevant data; The feedback presentation unit is used to feedback the interaction results to the user in the forms of vision, audition, and touch, provide operation confirmation, and guide the learning process.

[0010] Preferably, the network communication module includes a resource transmission module, a communication protocol adaptation module, a real-time communication module, and a network monitoring module; The resource transmission module is used to quickly and stably transmit various teaching resources stored on the server to the user terminal device; The communication protocol adaptation module is used to support common wired network protocols and wireless network protocols to ensure stable communication when the device is connected to the network; The real-time communication module is used to realize real-time message communication between users through the WebSocket real-time communication protocol, and to transmit the position, posture, and operation data information of users in real time, so as to keep each user terminal device participating in the interaction connected to the server. The server serves as a data transfer and coordination center to ensure that the AR scenes seen by all users are consistent; The network monitoring module is used to monitor the parameters of the network in real time, collect network data and analyze it, and timely discover network problems and performance bottlenecks.

[0011] Preferably, the multi-person AR interaction module includes a spatial positioning and synchronization module and an AR content sharing and synchronization update module; The spatial positioning and synchronization module is used to continuously perceive the environment through the device sensor by means of SLAM technology, construct a map of the surrounding environment. On this basis, according to feature matching and geometric calculation, the accurate position and posture of the device relative to the map are determined. At the same time, the position and posture information of each user device is summarized, and a unified spatial coordinate system is generated through calculation to ensure that the positions and directions of all users in the virtual space are consistent, so that the virtual content can be correctly and consistently superimposed on the real scene of each user; The AR content sharing and synchronization update module is used to manage the AR content stored and shared by the server. When a user operates on the virtual content, the server will record these operations in real time and push the updated content to the terminal devices of other users synchronously to realize the real-time sharing and synchronization update of the AR content.

[0012] Preferably, the spatial positioning and synchronization module includes environment perception and map construction, positioning and tracking, and spatial synchronization; The environment perception and map construction continuously collects image information of the surrounding environment through the sensor of the user terminal, measures the state data of the device in real time through the IMU, processes the collected images, extracts the feature points in the images, and combines the IMU data to calculate the movement trajectory of the device in space, and gradually constructs a map of the surrounding environment; The positioning and tracking is based on the constructed map and the currently collected image data, and determines the position and posture of the device relative to the map through feature matching and geometric calculation; Among them, the relationship between the posture and position change of the device in space is expressed by the following formula: ; Where: is an arbitrary point in the world coordinate system, then is the corresponding point in the device coordinate system, R is the rotation matrix, the attitude of the i-th user device is a 3X3 orthogonal matrix, satisfying, representing the rotation direction of the device, T represents the transpose of the matrix, and t is the translation vector which is a 3X1 vector, representing the translation amount of the device in space; The space synchronization is that the server calculates a unified space coordinate system according to the position and attitude information of all received user devices, ensures the consistency of the positions and directions of all users in the virtual space, and the server sends the synchronized space information back to each user device; Among them, the process of calculating the unified space coordinate system is as follows: ① Select a reference device m, and convert the positions and attitudes of other devices to the coordinate system based on the reference device; ② Then calculate the rotation difference matrix, the rotation difference matrix from device i to device m is: ; ③ Calculate the position difference vector, the position difference vector from device i to device m is , first convert the position vector of device i to the coordinate system of device m, and then calculate the difference from the position vector of device i, and the calculation formula is as follows: ; ④ Through the above calculations, for the i-th device, in the unified space coordinate system based on device m, its attitude is represented by the rotation difference matrix , and the position is represented by the position difference vector .

[0013] Preferably, the learning record and evaluation module includes a behavior record module, a data analysis module, and a learning evaluation module; The behavior record module is used to record various learning behaviors of students in the system, track the interaction operations of students with AR content, and save the homework and test answers submitted by students; The data analysis module is used to analyze the learning progress of students according to the learning behavior records of students, combine the homework and test scores of students, and the performance of interaction operations during the learning process, analyze the mastery of each knowledge point by students, and analyze the learning habits and preferences of students; There is a linear relationship between the learning time x and the number of completed course chapters y of the students in the learning progress analysis: ; where and is the parameter to be estimated, is the error term, and the parameters are estimated by the least squares method so that the error sum of squares Minimum, parameter estimation formula is: ; ; in, ; In the knowledge mastery analysis, for a question j, the probability that student i answers correctly is for: ; in, It is the topic differentiation. The difficulty of the question. is a guess parameter, is the student’s ability level; The learning assessment module is used to conduct real-time assessment of students' learning progress and performance based on learning records and data analysis, and generate a personalized learning assessment report for each student. The report content includes the student's learning goal achievement, knowledge mastery, and learning progress.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. In the present invention, the resource upload module facilitates teachers or administrators to actively upload teaching resources, continuously expand the scale of the resource library, and at the same time, strictly review the copyright and classify the uploaded resources, and add metadata, which helps to improve the standardization and availability of resources, cooperate with the teaching resource library to update regularly, add new content, correct error information, optimize resource quality, ensure that resources keep pace with the times and maintain high quality standards, and provide automatic resource labeling and classification description functions, so as to efficiently organize and classify teaching resources, and indirectly promote the improvement of resource quality; 2. In the present invention, by reasonably setting parameters and allocating resources according to teaching needs and equipment characteristics, adjusting resource loading and graphics rendering accuracy for different performance devices, reducing the number of model faces and texture resolution of low-configuration devices, reducing jamming, and optimizing AR content according to device screen resolution and display characteristics through a content adaptation module, ensuring clear display and appropriate proportions for different devices, solving problems such as fuzzy display, disproportionate proportions, and abnormal operation, optimizing AR content according to device screen resolution and display characteristics, ensuring clear display and appropriate proportions for different devices, and solving problems such as fuzzy display, disproportionate proportions, and abnormal operation; 3. In the present invention, by means of the SLAM technology, the device sensors continuously perceive the environment to construct a map of the surrounding environment. On this basis, according to feature matching and geometric calculation, the precise position and attitude of the device relative to the map are determined. Through environment perception and map construction, the surrounding environment image information and the IMU device status data are collected in real time, feature points are extracted and combined with the calculation of the device movement trajectory, so that the position of the device in space can be accurately determined, the positioning error is reduced, and the position and attitude information of each user device is calculated to generate a unified space coordinate system, ensuring that the positions and directions of all users in the virtual space are consistent, enabling the virtual content to be correctly and consistently superimposed on the real scene of each user, and improving the accuracy of space positioning; 4. In the present invention, through the AR content sharing and synchronous update module in the multi-person AR interaction module, when a user operates on the virtual content, the server will record these operations in real time and push the updated content to the terminal devices of other users synchronously. This enables students to see the operation results of each other in a timely manner during the multi-person interaction process, realizing real-time collaboration, solving the problem of poor collaboration caused by the inability to synchronize in real time during multi-person interaction, enhancing the teaching effect and the learning experience of students, and cooperating with the real-time communication technology to ensure the timely transmission of data during multi-person interaction, effectively reducing data latency, realizing real-time synchronization of multi-person interaction, and improving the students' collaborative learning experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a system block diagram of a teaching display system based on AR interaction technology of the present invention; Figure 2 is a system block diagram of the learning record and evaluation module of a teaching display system based on AR interaction technology of the present invention; Figure 3 is a flowchart of calculating a unified space coordinate system of a teaching display system based on AR interaction technology of the present invention.

[0016] In the figure: 1. User terminal device module; 2. Teaching resource module; 3. AR content generation module; 4. Interaction control module; 5. Network communication module; 6. Multi-person AR interaction module; 7. Learning record and evaluation module. DETAILED DESCRIPTION OF THE INVENTION

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] Refer to Figure 1 -Figure 3 As shown: A teaching display system based on AR interaction technology, including a user terminal device module 1, a teaching resource module 2, an AR content generation module 3, an interaction control module 4, a network communication module 5, a multi-person AR interaction module 6, and a learning record and evaluation module 7.

[0019] The user terminal device module 1 is used to manage various users and devices, including user registration, login, permission setting, and access management of multi-terminal devices, simplifying the user registration and login process, improving user operation convenience, and at the same time ensuring system data security and operation specifications through fine-grained permission management; specifically, it includes a user registration and login module, a permission management module, and a device configuration management module.

[0020] The user registration and login module is used to provide a registration entry for new users and allow users to input personal basic information to provide a user login function.

[0021] The permission management module is used to classify and manage the permissions and responsibilities of different users.

[0022] The device configuration management module is used to reasonably set the parameters of the device according to teaching needs and device characteristics, allocate device resources, and reasonably configure device parameters according to teaching scenarios and device performance to improve device usage efficiency and user experience.

[0023] The teaching resource module 2 is used for teachers to upload teaching resources and classify, store, retrieve, and update teaching resources, provide functions of automatic resource annotation and classification description, real-time statistics and update of teaching resources, and at the same time support teachers to create, edit, and manage AR teaching courses. Specifically, it includes a resource upload module, a retrieval and recommendation module, and an update and maintenance module.

[0024] The resource upload module is used for teachers or administrators to have the permission to upload teaching resources to the system, classify and manage the uploaded resources, review the copyright of the resources, add metadata to each teaching resource, and have efficient resource upload, classification, and retrieval functions, saving teachers' time for finding and managing resources.

[0025] The retrieval and recommendation module is used to provide a resource retrieval function and, based on the user's learning history, interest preferences, subject needs, etc., use algorithms to intelligently recommend relevant teaching resources to users.

[0026] The update and maintenance module is used to regularly update the teaching resource library, including adding new teaching content, correcting incorrect information, optimizing the quality of resources, intelligently recommending resources based on the user's learning situation, and ensuring the accurate push of resources; regular update and maintenance to ensure that the teaching content keeps up with the times and has high quality.

[0027] The AR content generation module 3 is used to convert various materials in the teaching resource module 2 in terms of resource format, coordinate and spatial positioning, material and texture processing, fusion and synthesis processing, and optimize and adapt according to the screen resolution, display characteristics, etc. of various terminal devices, ensuring that the AR teaching content is clearly displayed and proportionally appropriate on different devices. It uses AR technology to convert teaching resources into AR content, supports dynamic interaction functions for 3D models, optimizes 3D models to adapt to terminal devices, and improves rendering efficiency; accurately fuses with the real scene to enhance the realism of the combination of virtual and real; generates dynamic content and adapts content to meet diverse teaching needs and different device displays. Specifically, it includes a 3D model processing module, a real scene fusion module, a dynamic content generation module, and a content adaptation module.

[0028] The 3D model processing module is used to import and parse 3D models, ensure that the model format can be recognized and processed by the system, and then optimize the model to reduce the number of faces and texture size of the model, so as to improve the rendering efficiency and running smoothness on terminal devices; the model optimization processing includes face number simplification and texture compression.

[0029] For face number simplification, the quadratic error metric algorithm is adopted. For a triangular mesh model, each vertex v has an associated quadratic error matrix. When merging two vertices and into a new vertex , the error matrix of the new vertex , the goal is to find the vertex merging operation that minimizes the error. The error . By continuously merging vertices until the required number of faces is reached.

[0030] For texture compression, the DXT compression algorithm is adopted. The color of each pixel block is represented by the colors of two end points and a 2-bit index value. The 2-bit index value is the key information for determining the color of each pixel in a 4x4 pixel block. Specifically, for a 4x4 pixel block, DXT uses the colors of two end points to represent the color range of the block. Then, through the 2-bit index value, it is used to select which of the two end point colors to use, or a certain interpolation color between them to represent the color of each pixel. The color interpolation formula is: ; .

[0031] The real scene fusion module is used to accurately overlay virtual content at the appropriate position in the real scene according to the teaching needs and the characteristics of the real scene, and determine the position, angle and size of the virtual content by using the information provided by the scene recognition and tracking module.

[0032] The dynamic content generation module is used to generate dynamic AR content in real time according to the teaching logic and the user's interaction operations.

[0033] The content adaptation module is used to adapt and optimize the generated AR content, and adjust the resolution, display ratio and interaction mode of the content.

[0034] The interaction control module 4 realizes the interaction between the user and the virtual content in the teaching display system based on AR interaction technology, supports multiple interaction methods, such as gesture recognition, voice control, somatosensory interaction, etc. The multiple interaction methods broaden the user's operation channels, accurately identify and process operation instructions, and timely feedback results, enhancing the user's sense of participation and learning enthusiasm; specifically, it includes an interaction method recognition unit, an interaction logic processing unit and a feedback presentation unit.

[0035] The interaction method recognition unit is used to capture the user's action input by using various sensors, such as cameras, microphones, depth cameras, etc., convert the user's actual operations into signals that the system can process, analyze the operation intention and provide an interaction basis.

[0036] The interaction logic processing unit is used to receive and convert the user instructions from the interaction method recognition unit, judge and control the user's operations according to the interaction rules and teaching objectives, and manage and coordinate relevant data.

[0037] The feedback presentation unit is used to feedback the interaction results to the user in the forms of vision, audition and touch, provide operation confirmation, and guide the learning process.

[0038] The network communication module 5 is used to realize the communication between various modules in the system and between the system and external devices, and fast resource transmission, ensure the timely acquisition of teaching resources, adapt to different network environments through multi-protocol adaptation, promote interactive communication through real-time communication, and maintain the stable operation of the system through network monitoring; specifically, it includes a resource transmission module, a communication protocol adaptation module, a real-time communication module and a network monitoring module.

[0039] The resource transmission module is used to quickly and stably transmit various teaching resources stored on the server to the user terminal device.

[0040] The communication protocol adaptation module is used to support common wired network protocols and wireless network protocols to ensure stable communication when the device is connected to the network.

[0041] The real-time communication module is used to achieve real-time message communication between users through the WebSocket real-time communication protocol, and transmit data information such as the location, posture, and operations of users in real time. Each user terminal device participating in the interaction needs to be connected to the server. The server serves as a data transfer and coordination center, receives and processes information from each terminal, and then synchronously sends relevant data to other terminals to ensure that the AR scenes seen by all users are consistent.

[0042] The network monitoring module is used to monitor parameters such as the connection status, bandwidth usage, and latency of the network in real time, collect network data and perform analysis, and timely discover network problems and performance bottlenecks.

[0043] The multi-user AR interaction module 6 is used for multiple users to participate in AR teaching activities simultaneously, and realizes interaction and collaboration between users, creates a multi-user shared AR virtual environment, enables different users to interact in the same virtual space through spatial synchronous positioning, realizes AR content sharing and synchronous update, accurate spatial positioning and synchronization, and ensures the consistency of the multi-user AR scene; AR content sharing and update realizes real-time collaboration among students and cultivates the ability of teamwork and knowledge sharing. Specifically, it includes a spatial positioning and synchronization module and an AR content sharing and synchronous update module.

[0044] The spatial positioning and synchronization module is used to continuously perceive the environment through device sensors with the help of SLAM technology, construct a map of the surrounding environment. On this basis, based on feature matching and geometric calculations, it realizes the accurate determination of the position and posture of the device relative to the map. At the same time, it summarizes the position and posture information of each user device, calculates and generates a unified spatial coordinate system to ensure that the positions and directions of all users in the virtual space are consistent, so that the virtual content can be correctly and consistently superimposed on the real scene of each user, and the relative position relationship between each other remains accurate; it includes environment perception and map construction, positioning and tracking, and spatial synchronization.

[0045] The environment perception and map construction continuously collects image information of the surrounding environment through the sensors of the user terminal, measures the state data of the device in real time through the IMU, processes the collected images, extracts the feature points in the images, combines the IMU data to calculate the movement trajectory of the device in space, and gradually constructs a map of the surrounding environment.

[0046] The positioning and tracking is based on the constructed map and the currently collected image data, and determines the position and posture of the device relative to the map through feature matching and geometric calculations; Among them, the relationship between the posture and position changes of the device in space is expressed by the following formula: ; In the formula: is an arbitrary point in the world coordinate system, then is the corresponding point in the device coordinate system, R is the rotation matrix, the pose of the i-th user device is a 3X3 orthogonal matrix, satisfying, represents the rotation direction of the device, T represents the transpose of the matrix, and t is the translation vector which is a 3X1 vector representing the translation amount of the device in space.

[0047] Spatial synchronization is achieved by the server calculating a unified spatial coordinate system based on the received position and pose information of all user devices, ensuring the consistency of the positions and directions of all users in the virtual space. The server then sends the synchronized spatial information back to each user device. Among them, the process of calculating the unified spatial coordinate system is as follows: ① Select a reference device m and convert the positions and poses of other devices to the coordinate system with the reference device as the benchmark. ② Then calculate the rotation difference matrix, the rotation difference matrix from device i to device m is: ; ③ Calculate the position difference vector, the position difference vector from device i to device m is , first convert the position vector of device i to the coordinate system of device m, and then calculate the difference from the position vector of device i. The calculation formula is as follows: ; ④ Through the above calculations, for the i-th device, in the unified spatial coordinate system with device m as the benchmark, its pose is represented by the rotation difference matrix , and the position is represented by the position difference vector .

[0048] The AR content sharing and synchronization update module is used to manage the server to store and manage the shared AR content. When a user operates on the virtual content, the server will record these operations in real time and push the updated content to the terminal devices of other users to achieve real-time sharing and synchronization update of the AR content.

[0049] The learning record and evaluation module 7 is used to record the learning activities and related data of students in the system, analyze based on the learning data to evaluate the learning effect of students, provide teaching feedback for teachers, and also provide learning suggestions and improvement directions for students. By comprehensively recording learning behaviors and deeply analyzing learning data, it provides accurate teaching feedback for teachers to assist in teaching strategy adjustment; generates personalized reports for students to guide learning improvement and development; specifically, it includes a behavior recording module, a data analysis module, and a learning evaluation module.

[0050] The behavior recording module is used to record various learning behaviors of students in the system, track students' interaction operations with AR content, and save the homework and test answers submitted by students.

[0051] The data analysis module is used to analyze students' learning progress based on their learning behavior records, and combine students' homework and test scores, as well as their performance in interaction operations during the learning process, to analyze students' mastery of each knowledge point and their learning habits and preferences.

[0052] There is a linear relationship between the learning time x and the number of completed course chapters y of students in the learning progress analysis: ; where and are parameters to be estimated, is the error term, and the parameters are estimated by the least squares method to minimize the sum of squared errors . The parameter estimation formula is: ; ; where, ; In the analysis of the degree of knowledge mastery, for a question j, the probability that student i answers correctly is: ; where, is the item discrimination, is the item difficulty, is the guessing parameter, is the ability level of the student.

[0053] The learning evaluation module is used to conduct real-time evaluation of students' learning progress and performance based on learning records and data analysis, and generate personalized learning evaluation reports for each student. The report content includes the achievement of students' learning goals, the degree of knowledge mastery, and the learning progress.

[0054] In the present invention, it supports teachers to upload various teaching resources, and finely manage the resources, including classification, retrieval, recommendation, update and maintenance, and provides the functions of course creation and editing, greatly enriching the sources of teaching materials and the flexibility of course design. By converting teaching resources into interactive AR content, from 3D model processing, real-scene fusion to dynamic content generation and content adaptation, it creates an immersive AR teaching experience in all aspects. It recognizes user operations through various interaction methods, processes instructions according to rules, and timely feedbacks interaction results, improving the natural fluency and interest of the user's interaction with virtual content. With the help of SLAM technology, it realizes spatial positioning synchronization and AR content sharing and update, promotes students to collaborate in the same AR scene, and cultivates their cooperation and communication abilities. By recording students' learning behaviors and analyzing learning data, it evaluates students' learning effects and generates personalized reports, providing a strong basis for teaching improvement and students' development.

[0055] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A teaching display system based on AR interaction technology, characterized in that: It includes a user terminal device module (1), a teaching resource module (2), an AR content generation module (3), an interaction control module (4), a network communication module (5), a multi-person AR interaction module (6), and a learning record and evaluation module (7); The user terminal device module (1) is used to manage various users and devices, including user registration, login, permission setting, and access management of multi-terminal devices; The teaching resource module (2) is used for teachers to upload teaching resources and classify, store, retrieve, and update the teaching resources, provide functions of automatic resource annotation and classification description, statistically and update the situation of teaching resources in real time, and at the same time support teachers to create, edit, and manage AR teaching courses; The AR content generation module (3) is used to process and process various materials in the teaching resource module (2), and use AR technology to convert teaching resources into AR content, and support dynamic interaction functions for 3D models; The interaction control module (4) realizes the interaction between users and virtual content in the teaching display system based on AR interaction technology, and supports multiple interaction methods; The network communication module (5) is used to realize the communication between various modules in the system and between the system and external devices; The multi-person AR interaction module (6) is used for multiple users to participate in AR teaching activities at the same time, and realizes the interaction and cooperation between users, creates a multi-person shared AR virtual environment, and enables different users to interact in the same virtual space through spatial synchronous positioning, realizing AR content sharing and synchronous update; The learning record and evaluation module (7) is used to record the learning activities and related data of students in the system, analyze according to the learning data to evaluate the learning effect of students, provide teaching feedback for teachers, and also provide learning suggestions and improvement directions for students.

2. The teaching display system based on AR interaction technology according to claim 1, wherein: The user terminal device module (1) includes a user registration and login module, a permission management module, and a device configuration management module; The user registration and login module is used to provide a registration entry for new users, and support users to input personal basic information to provide user login functions; The permission management module is used to classify and manage the permissions and responsibilities of different users; The device configuration management module is used to set the parameters of the device according to teaching requirements and device characteristics, and allocate device resources.

3. The teaching display system based on AR interaction technology according to claim 1, wherein: The teaching resource module (2) includes a resource upload module, a retrieval and recommendation module, and an update and maintenance module; The resource upload module is used for teachers or administrators to have the permission to upload teaching resources to the system, and at the same time classify and manage the uploaded resources, review the copyright of the resources, and add metadata to each teaching resource; The retrieval and recommendation module is used to provide a resource retrieval function, and use algorithms to intelligently recommend relevant teaching resources for users according to the user's preference data; The update and maintenance module is used to regularly update the teaching resource library, including adding new teaching content, correcting incorrect information, and optimizing the quality of resources.

4. A teaching display system based on AR interaction technology according to claim 1, characterized in that: The AR content generation module (3) includes a 3D model processing module, a real-world scene fusion module, a dynamic content generation module, and a content adaptation module; The 3D model processing module is used to import and parse 3D models, ensure that the model formats can be recognized and processed by the system, and optimize the models. The real-scene fusion module is used to overlay virtual content at appropriate positions in the real scene according to teaching requirements and the characteristics of the real scene, and determine the position, angle, and size of the virtual content by using the information provided by the scene recognition and tracking module. The dynamic content generation module is used to generate dynamic AR content in real time according to teaching logic and users' interactive operations. The content adaptation module is used to adapt and optimize the generated AR content, and adjust the resolution, display ratio, and interaction method of the content.

5. A teaching display system based on AR interaction technology according to claim 1, characterized in that: The interaction control module (4) includes an interaction method recognition unit, an interaction logic processing unit, and a feedback presentation unit. The interaction method recognition unit is used to capture users' inputs by using various sensors, convert users' actual operations into signals that can be processed by the system, analyze the operation intentions, and provide an interaction basis. The interaction logic processing unit is used to receive and convert users' instructions from the interaction method recognition unit, judge and control users' operations according to interaction rules and teaching objectives, and manage and coordinate relevant data. The feedback presentation unit is used to feedback the interaction results to users in the forms of vision, audition, and touch, provide operation confirmation, and guide the learning process.

6. The teaching display system based on AR interaction technology according to claim 1, characterized in that: The network communication module (5) includes a resource transmission module, a communication protocol adaptation module, a real-time communication module, and a network monitoring module. The resource transmission module is used to quickly and stably transmit various teaching resources stored on the server to user terminal devices. The communication protocol adaptation module is used to support common wired network protocols and wireless network protocols to ensure stable communication when the device is connected to the network. The real-time communication module is used to realize real-time message communication between users through the WebSocket real-time communication protocol, and real-time transmit users' position, attitude, and operation data information, keep each user terminal device participating in the interaction connected to the server, and the server serves as a data transfer and coordination center to ensure that the AR scenes seen by all users are consistent. The network monitoring module is used to monitor network parameters in real time, collect network data and analyze it, and timely discover network problems and performance bottlenecks.

7. An instructional display system based on AR interaction technology according to claim 1, wherein: The multi-person AR interaction module (6) includes a space positioning and synchronization module and an AR content sharing and synchronization update module. The space positioning and synchronization module is used to continuously perceive the environment through device sensors by means of SLAM technology, construct a map of the surrounding environment, and on this basis, determine the precise position and attitude of the device relative to the map according to feature matching and geometric calculation. At the same time, summarize the position and attitude information of each user device, and generate a unified space coordinate system through calculation to ensure that the positions and directions of all users in the virtual space are consistent, so that the virtual content can be correctly and consistently overlaid on the real scenes of each user. The AR content sharing and synchronous update module is used to manage the AR content stored and shared by the server. When a user operates on virtual content, the server will record these operations in real time and push the updated content to the terminal devices of other users synchronously, realizing the real-time sharing and synchronous update of AR content.

8. A teaching display system based on AR interaction technology according to claim 7, characterized in that: The spatial positioning and synchronization module includes environment perception and map construction, positioning and tracking, and spatial synchronization; The environment perception and map construction continuously collects image information of the surrounding environment through the sensors of the user terminal, measures the state data of the device in real time through the IMU, processes the collected images, extracts the feature points in the images, combines the IMU data to calculate the movement trajectory of the device in space, and gradually constructs a map of the surrounding environment; The positioning and tracking is based on the constructed map and the currently collected image data, and determines the position and attitude of the device relative to the map through feature matching and geometric calculation; The spatial synchronization calculates a unified spatial coordinate system by the server according to the position and attitude information of all user devices received, ensuring the consistency of the positions and directions of all users in the virtual space. The server sends the synchronized spatial information back to each user device.

9. A teaching display system based on AR interaction technology according to claim 1, characterized in that: The learning record and evaluation module (7) includes a behavior record module, a data analysis module, and a learning evaluation module; The behavior record module is used to record various learning behaviors of students in the system, track the interaction operations of students with AR content, and save the homework and test answers submitted by students; The data analysis module is used to analyze the learning progress of students according to the learning behavior records of students, and combine the homework and test scores of students, as well as the performance of interaction operations in the learning process, to analyze the mastery of each knowledge point by students, and analyze the learning habits and preferences of students; The learning evaluation module is used to conduct real-time evaluation of the learning progress and performance of students according to the learning records and data analysis, and generate a personalized learning evaluation report for each student. The report content includes the achievement of students' learning goals, the degree of knowledge mastery, and the learning progress.

Images