Teaching system based on AR interaction

Abstract

The invention provides a teaching system based on AR interaction, and relates to the technical field of intelligent teaching, and the system comprises a data obtaining module which is used for obtaining the environment data of a real scene in real time, outputting the real-time scene data, and obtaining the type of a teaching task and the simulation scene data of the corresponding type; the data conversion module is used for performing data conversion on the real-time scene data, outputting scene conversion data and matching the scene conversion data with the simulation scene data; the virtual reality combination module is used for projecting the simulation scene data to a real scene corresponding to the scene conversion data; and the interaction module is used for acquiring an interaction instruction of a user, adjusting and controlling adjustable parameters in the simulation scene data, and dynamically adjusting the simulation scene data in combination with variation parameters of the adjustable parameters. The stability and real-time performance of the AR interaction system in the teaching task are improved, normal proceeding of the teaching task is guaranteed, and the actual use experience of teachers and students is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent teaching technology, and in particular to a teaching system based on AR interaction. Background Art

[0002] The teaching system based on augmented reality (AR) interaction is an innovative way to improve teaching methods using AR technology. This system can provide an immersive learning experience, allowing students to interact with virtual objects to enhance their understanding and memory of knowledge. At the same time, it can transform abstract concepts into visual content, making learning more specific and intuitive, helping students to better understand and master knowledge. It can also provide virtual practice and simulation environments, allowing students to practice and try in a safe environment to improve their practical operation capabilities.

[0003] At present, although this highly interactive teaching system can greatly improve the teaching quality, there are still some challenges in the stability and real-time performance of AR technology in actual teaching tasks, which may cause system instability and delays and affect the user experience.

[0004] Therefore, the present invention provides a teaching system based on AR interaction. Summary of the invention

[0005] The present invention provides a teaching system based on AR interaction, which is used to improve the stability and real-time performance of the system in teaching tasks, ensure the normal progress of teaching tasks, and improve the actual use experience of teachers and students.

[0006] The present invention provides a teaching system based on AR interaction, comprising: The data acquisition module is used to acquire the environmental data of the real scene in real time by using the preset scanning device, output the real-time scene data, and at the same time, acquire the type of teaching task and the simulation scene data under the corresponding type; A data conversion module is used to perform data conversion on the real-time scene data using a preset scene construction algorithm, output scene conversion data, and update the scene database, and at the same time, match the scene conversion data with the simulation scene data; A virtual reality combination module is used to project the simulated scene data that meets the preset matching conditions into the real scene corresponding to the scene conversion data; The interactive module is used to obtain the user's interactive instructions in real time, and adjust and control the adjustable parameters in the simulation scene data based on the interactive instructions, and at the same time, dynamically adjust the corresponding simulation scene data in combination with the variable parameters of the adjustable parameters.

[0007] Preferably, the data acquisition module includes: A real scene scanning submodule, used to use the preset scanning device to perform a three-dimensional scan of the real scene environment around the device in real time, and to construct a scene based on the scanned data to obtain real-time scene data; The virtual scene acquisition submodule is used to acquire the type of teaching task and acquire the corresponding simulation scene data under the type of teaching task in the scene database.

[0008] Preferably, the real scene scanning submodule includes: A first solution building unit, configured to scan a real scene around the device at different positions using the same preset scanning device, and output a first scanning solution; A second scheme constructing unit, configured to construct a scanning device array according to at least two preset scanning devices at multiple different positions, and output a second scanning scheme; The data scanning unit is used to obtain three-dimensional data of the real environment around the device in real time based on the first scanning scheme and the second scanning scheme through a preset scanning device, and to perform scene construction to obtain real-time scene data.

[0009] Preferably, the data scanning unit comprises: A scheme determination subunit is used to calculate the complexity of the target real scene according to a preset algorithm, and compare and analyze it with a preset threshold condition. When the scene complexity is less than a first preset complexity, a first scanning scheme is adopted, otherwise, a second scanning scheme is adopted; The data scanning subunit is used to determine the scanning scheme adopted by the subunit based on the scheme, and to obtain the three-dimensional data of the real environment around the device in real time through a preset scanning device, and to construct the scene to obtain real-time scene data.

[0010] Preferably, the data conversion module includes: A scene conversion submodule, used to perform data conversion on the real-time scene data according to the preset scene construction algorithm, and output scene conversion data; A data updating submodule, used for comparing and analyzing the scene conversion data with the constructed scene data in the scene database, and determining whether to update the scene conversion data to the scene database based on the comparison and analysis result; The data matching submodule is used to perform three-dimensional matching between the scene conversion data and the simulation scene data, and determine the projection parameters of the simulation scene data based on the matching result.

[0011] Preferably, the data updating submodule includes: A data analysis unit, used to analyze the similarity and difference between the scene conversion data and the constructed scene data in the scene database; The update determination unit is used to compare and analyze the similarity and difference with the preset data update threshold condition, upload and update the scene conversion data that meets the preset data update threshold condition to the scene database, otherwise, the scene conversion data is not updated to the scene database.

[0012] Preferably, the virtual reality combining module comprises: A projection position selection submodule, used to determine a position area for projection by a projection device in a real scene based on the real-time scene data and in combination with a user's position selection instruction; wherein, the scene conversion data is analyzed, and an area where the analysis result satisfies a preset projection condition is output as a projection-enabled area; Obtaining a location selection instruction from a user, and selecting an area to be projected from each of the projection-enabled areas; A content acquisition submodule, used to determine the projection content and projection dimension in the to-be-projected area based on the type of teaching task and the simulation scene data under the corresponding type; A device selection submodule, used to determine the number of projection devices to be called based on the projection content and projection dimension, and determine the projection distance, projection angle and projection sub-content of each projection device; The calibration submodule is used to calibrate and adjust each projection device before projection, and determine the projection parameters of each projection device; A projection submodule, configured to project the projection sub-content of the corresponding device into the to-be-projected area based on the projection of each projection device; An adaptive submodule, used to obtain environmental change parameters in real time during the projection process, and to adaptively adjust the projection parameters of each projection device in real time based on the environmental change data, wherein the environmental change parameters include light parameters near the projection area, interference source parameters, and communication parameters of each projection device; The function display submodule is used to obtain the interactive items and corresponding interactive parameters corresponding to the current real-time display screen in the projection content, and display them in real time in a preset area in the area to be projected, and at the same time, mark the interactive items with information and generate interactive prompt information; The self-service submodule is used to provide the user with an operation prompt manual corresponding to the current real-time display screen.

[0013] Preferably, the interactive module includes: The motion capture submodule is used to capture the user's body movements in real time through a preset motion capture device to generate motion capture data; A limb motion analysis submodule, used for analyzing the motion capture data to generate motion capture analysis data; A trigger determination submodule is used to perform posture analysis on the motion capture analysis data in combination with a preset classification decision algorithm, determine the user's action type and output the posture analysis result; ; in, Represents the i-th posture judgment value, when When , it indicates that the user's action type is the i-th action posture. When , it indicates that the user's action type is not the i-th action posture; represents the exponential function; Representation and The error adjustment factor corresponding to the data point; represents the function used to make classification decisions on the trajectories of the motion capture data; represents the total number of data points in the training data set corresponding to the i-th posture in the preset classification decision algorithm; represents the Langrange multiplier corresponding to the jth data point; Indicates The category labels corresponding to the training samples; Indicates the calculation for training samples and samples to be analyzed in motion capture analysis data The kernel function of the similarity between them; represents the bias term used to adjust the decision boundary; represents the threshold used for posture determination of the i-th posture; An interactive instruction generation submodule is used to verify and compare the gesture analysis result with the adjustable parameters in the simulation scene data at the same time to determine the interactive instruction input by the user; An instruction execution submodule, used to adjust and control the adjustable parameters in the simulation scene data based on the interactive instruction, and output parameter change data; A projection parameter adjustment submodule, used to adjust the projection parameters of the projection device based on the parameter change data, and dynamically adjust the simulation scene data; The effect evaluation submodule is used to obtain user feedback data, evaluate the interaction effect based on preset evaluation indicators, and output the effect analysis results; The system optimization submodule is used to optimize the interactive teaching process based on the effect analysis results and in combination with preset optimization goals.

[0014] The present invention provides an AR interactive teaching system, which acquires real-time scene data through a preset scanning device, converts the real-time scene data into virtual scene data using a preset scene construction algorithm, and realizes the interaction between the user and the virtual scene in combination with an interactive module. The present invention can provide a more specific and intuitive learning experience, enhance students' learning interest and participation, and promote the understanding and memory of knowledge. At the same time, by dynamically adjusting adjustable parameters, users can personalize the virtual scene according to their own needs to achieve better learning effects and experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a schematic diagram of the framework of a teaching system based on AR interaction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, an embodiment of the present invention provides a teaching system based on AR interaction, including: The data acquisition module is used to acquire the environmental data of the real scene in real time by using the preset scanning device, output the real-time scene data, and at the same time, acquire the type of teaching task and the simulation scene data under the corresponding type; A data conversion module is used to perform data conversion on the real-time scene data using a preset scene construction algorithm, output scene conversion data, and update the scene database, and at the same time, match the scene conversion data with the simulation scene data; A virtual reality combination module is used to project the simulated scene data that meets the preset matching conditions into the real scene corresponding to the scene conversion data; The interactive module is used to obtain the user's interactive instructions in real time, and adjust and control the adjustable parameters in the simulation scene data based on the interactive instructions, and at the same time, dynamically adjust the corresponding simulation scene data in combination with the variable parameters of the adjustable parameters.

[0019] In this embodiment, the preset scanning device is a pre-configured device for acquiring real scene environment data, such as AR glasses or AR cameras, specifically, AR devices such as Microsoft HoloLens and Magic Leap; In this embodiment, environmental data refers to data of a real scene obtained from a preset scanning device, including information such as objects, structures, and colors in the scene, for example, actual scene data inside a classroom obtained from an AR camera; In this embodiment, real-time scene data: real-time environmental data collected and processed by a preset scanning device; In this embodiment, the types of teaching tasks include: different teaching objectives or task types set in the system, such as mathematics tasks, physical experiments, historical scene reproduction, etc.; In this embodiment, simulation scene data: virtual scene data prepared in advance according to the type of teaching task, virtualized chemical laboratory, ancient battlefield reproduction, etc.; In this embodiment, the preset scene construction algorithm: an algorithm for converting real-time scene data into virtual scene data, such as a deep learning algorithm, a computer vision algorithm, etc.; In this embodiment, scene conversion data: the result of real-time scene data conversion after being processed by a preset scene construction algorithm, for example, data for converting a real-time classroom scene into a virtual laboratory scene; In this embodiment, the scene database: a database storing a large amount of simulation scene data, for example, a cloud database, etc.; In this embodiment, the matching conditions are preset: conditions for determining which simulated scene data should be projected into the real scene, for example, when specific information or objects appear, the corresponding virtual information projection is triggered; In this embodiment, interactive instruction refers to an instruction for a user to interact with a virtual scene through the system, such as gestures, voice or button clicks; In this embodiment, adjustable parameters: parameters in the simulated scene data that can be adjusted and controlled according to user operations, such as the size of the simulated object and the light intensity, object color, scene speed, etc. in the simulated scene; The implementation principle and beneficial effects of this embodiment: The present invention obtains real-time scene data through a preset scanning device, converts the real-time scene data into virtual scene data using a preset scene construction algorithm, and realizes the interaction between the user and the virtual scene in combination with an interactive module. The present invention can provide a more specific and intuitive learning experience, enhance students' learning interest and participation, and promote the understanding and memory of knowledge. At the same time, by dynamically adjusting adjustable parameters, users can personalize the virtual scene according to their own needs to achieve better learning effects and experience.

[0020] An embodiment of the present invention provides an AR interactive teaching system, wherein the data acquisition module includes: A real scene scanning submodule, used to use the preset scanning device to perform a three-dimensional scan of the real scene environment around the device in real time, and to construct a scene based on the scanned data to obtain real-time scene data; The virtual scene acquisition submodule is used to acquire the type of teaching task and acquire the corresponding simulation scene data under the type of teaching task in the scene database.

[0021] In this embodiment, three-dimensional scanning: the process of collecting three-dimensional data of the real scene environment around the device using a preset scanning device, which can capture detailed information such as the geometric shape, color and texture of the scene for subsequent scene construction and virtualization. For example, a laser scanner is used to perform a three-dimensional scan of a classroom to obtain the geometric shape and structural information inside the classroom; The implementation principle and beneficial effects of this embodiment: The present invention performs a three-dimensional scan of the surrounding environment in real time through the real scene scanning submodule to obtain real scene data, and obtains corresponding simulated scene data through the virtual scene acquisition submodule. The present invention can combine the real world with the virtual world to provide students with an immersive learning experience.

[0022] An embodiment of the present invention provides an AR interactive teaching system, wherein the real scene scanning submodule includes: A first solution building unit, configured to scan a real scene around the device at different positions using the same preset scanning device, and output a first scanning solution; A second scheme constructing unit, configured to construct a scanning device array according to at least two preset scanning devices at multiple different positions, and output a second scanning scheme; The data scanning unit is used to obtain three-dimensional data of the real environment around the device in real time based on the first scanning scheme and the second scanning scheme through a preset scanning device, and to perform scene construction to obtain real-time scene data.

[0023] In this embodiment, the first scanning solution: This solution can help the system obtain scene data at different angles and viewing angles through a single device, with a small number of devices called, simple operation but complex process. For example, a single laser scanner is used to scan the classroom at different positions to obtain scene data at different angles and positions; In this embodiment, the scanning device array is a scanning device set constructed based on at least two preset scanning devices at multiple different positions. This device array can provide more comprehensive and three-dimensional scene data. For example, multiple laser scanners are used to form a scanning device array at different positions inside the classroom to obtain more dimensional scene information. In this embodiment, the second scanning scheme: this scheme combines data from multiple devices, has high scanning accuracy, and can provide richer and more accurate scene information.

[0024] The implementation principle and beneficial effects of this embodiment: The present invention constructs different scanning schemes through the first scheme construction unit and the second scheme construction unit, and then obtains the three-dimensional data of the real environment around the device in real time based on these schemes through the data scanning unit, and performs scene construction to obtain real-time scene data. This method of combining multiple schemes can provide users with different scanning schemes while ensuring the richness and accuracy of scene data, greatly improving the applicability of AR devices.

[0025] An embodiment of the present invention provides an AR interactive teaching system, wherein the data scanning unit includes: A scheme determination subunit is used to calculate the complexity of the target real scene according to a preset algorithm, and compare and analyze it with a preset threshold condition. When the scene complexity is less than a first preset complexity, a first scanning scheme is adopted, otherwise, a second scanning scheme is adopted; The data scanning subunit is used to determine the scanning scheme adopted by the subunit based on the scheme, and to obtain the three-dimensional data of the real environment around the device in real time through a preset scanning device, and to construct the scene to obtain real-time scene data.

[0026] In this embodiment, the preset algorithm: an algorithm preset by the system for calculating the complexity of the target real scene, used to evaluate the complexity of the scene and make a decision on whether to adopt the first scanning scheme or the second scanning scheme; In this embodiment, complexity refers to the complexity of the real scene, which can be measured according to various factors in the scene, such as the number of objects, shape complexity, texture diversity, etc. For example, a classroom scene may have a lower complexity because it usually contains basic elements such as tables, chairs, and blackboards; while a laboratory scene may have a higher complexity because it may contain various experimental equipment and devices; In this embodiment, preset threshold condition: a condition preset by the system, used to determine whether the complexity of the scene requires the first scanning scheme or the second scanning scheme; In this embodiment, the first preset complexity refers to a threshold value preset by the system and is used to determine whether to adopt the first scanning scheme. When the complexity of the target real scene is less than the first preset complexity, the system chooses to adopt the first scanning scheme, otherwise it adopts the second scanning scheme.

[0027] The implementation principle and beneficial effects of this embodiment: The present invention determines the complexity of the target real scene according to a preset algorithm through a scheme determination subunit, and compares and analyzes it with a preset threshold condition, thereby deciding to adopt the first scanning scheme or the second scanning scheme. Then, the data scanning subunit obtains the three-dimensional data of the real environment around the device in real time according to this decision, and performs scene construction to obtain real-time scene data. The present invention can flexibly adjust the scanning strategy according to the complexity of the scene, intelligently select the appropriate scanning scheme, improve the adaptability of the system, and at the same time, save the use of scanning equipment and system resources, and improve the accuracy and completeness of scene data.

[0028] An embodiment of the present invention provides an AR interactive teaching system, wherein the data conversion module includes: A scene conversion submodule, used to perform data conversion on the real-time scene data according to the preset scene construction algorithm, and output scene conversion data; A data updating submodule, used for comparing and analyzing the scene conversion data with the constructed scene data in the scene database, and determining whether to update the scene conversion data to the scene database based on the comparison and analysis result; The data matching submodule is used to perform three-dimensional matching between the scene conversion data and the simulation scene data, and determine the projection parameters of the simulation scene data based on the matching result.

[0029] In this embodiment, the constructed scene data refers to the scene data that already exists in the system, which may include three-dimensional models and related information of various scenes such as classrooms and laboratories. For example, the constructed scene data may be a three-dimensional model of a classroom, including the position, size and attribute information of elements such as tables and chairs, blackboards, and projectors; In this embodiment, three-dimensional matching refers to matching the scene conversion data with the simulation scene data in three-dimensional space to determine the adaptation effect between them and provide support for subsequent determination of projection parameters; In this embodiment, projection parameters are parameters used to describe the projection effect in the virtual scene, including viewing angle, lighting, perspective and other factors that affect the presentation of the virtual scene. For example, in a virtual laboratory scene, projection parameters may include parameters such as the position of the camera, viewing angle, direction and intensity of the light source, so as to ensure the realism and fidelity of the virtual scene.

[0030] The implementation principle and beneficial effects of this embodiment: The present invention uses a scene conversion submodule to perform data conversion on real-time scene data according to a preset scene construction algorithm and outputs scene conversion data; the data update submodule compares and analyzes the scene conversion data with the constructed scene data to determine whether to update the scene database; the data matching submodule performs three-dimensional matching of the scene conversion data with the simulated scene data to determine the projection parameters of the simulated scene data. Through these steps, the present invention can help the system dynamically update the scene database and adjust the projection parameters of the virtual scene to achieve more accurate virtual scene presentation.

[0031] An embodiment of the present invention provides an AR interaction-based teaching system, wherein the data updating submodule includes: A data analysis unit, used to analyze the similarity and difference between the scene conversion data and the constructed scene data in the scene database; The update determination unit is used to compare and analyze the similarity and difference with the preset data update threshold condition, upload and update the scene conversion data that meets the preset data update threshold condition to the scene database, otherwise, the scene conversion data is not updated to the scene database.

[0032] In this embodiment, similarity refers to the similarity or matching degree between the scene conversion data and the constructed scene data in the scene database. For example, in a teaching system, the similarity can be evaluated by comparing the elements, structure, attributes, etc. of the real-time scene data and the constructed scene data; In this embodiment, the difference degree: represents the degree of difference or mismatch between the scene conversion data and the constructed scene data, which can reflect the changes, missing or wrong information between the real-time scene data and the constructed scene data, and helps to determine the part that needs to be updated; In this embodiment, the data update threshold condition is preset: a condition preset by the system, used to determine whether the scene conversion data needs to be updated to the scene database, which can be a combination of a specific similarity threshold and a difference threshold. When the similarity is higher than a certain threshold and the difference is lower than another threshold, the condition is met and the data can be updated.

[0033] The implementation principle and beneficial effects of this embodiment: The present invention analyzes the similarity and difference between the scene conversion data and the constructed scene data through a data analysis unit, and then compares and analyzes the similarity and difference with the preset data update threshold conditions through an update determination unit. If the preset data update threshold conditions are met, that is, the similarity is higher than the threshold and the difference is lower than the threshold, the scene conversion data is uploaded and updated to the scene database; otherwise, no update operation is performed. The present invention can more accurately determine which parts need to be updated, improve the accuracy of data updates, avoid unnecessary data uploads and update operations, save system resources and improve the operating efficiency of the system, reduce the need for manual intervention, and improve the overall performance of the system.

[0034] An embodiment of the present invention provides a teaching system based on AR interaction, wherein the virtual reality combination module includes: A projection position selection submodule, used to determine a position area for projection by a projection device in a real scene based on the real-time scene data and in combination with a user's position selection instruction; wherein, the scene conversion data is analyzed, and an area where the analysis result satisfies a preset projection condition is output as a projection-enabled area; Obtaining a location selection instruction from a user, and selecting an area to be projected from each of the projection-enabled areas; A content acquisition submodule, used to determine the projection content and projection dimension in the to-be-projected area based on the type of teaching task and the simulation scene data under the corresponding type; A device selection submodule, used to determine the number of projection devices to be called based on the projection content and projection dimension, and determine the projection distance, projection angle and projection sub-content of each projection device; The calibration submodule is used to calibrate and adjust each projection device before projection, and determine the projection parameters of each projection device; A projection submodule, configured to project the projection sub-content of the corresponding device into the to-be-projected area based on the projection of each projection device; An adaptive submodule, used to obtain environmental change parameters in real time during the projection process, and to adaptively adjust the projection parameters of each projection device in real time based on the environmental change data, wherein the environmental change parameters include light parameters near the projection area, interference source parameters, and communication parameters of each projection device; The function display submodule is used to obtain the interactive items and corresponding interactive parameters corresponding to the current real-time display screen in the projection content, and display them in real time in a preset area in the area to be projected, and at the same time, mark the interactive items with information and generate interactive prompt information; The self-service submodule is used to provide the user with an operation prompt manual corresponding to the current real-time display screen.

[0035] In this embodiment, the position selection instruction is an instruction provided by the user, which is used to determine the position area for the projection device to perform projection in the real scene. The user may specify the projection position through gestures, voice or interface operations, for example, selecting a blackboard as the projection position in a classroom; In this embodiment, the preset projection conditions are conditions preset by the system to determine which areas can be projected, including conditions such as scene brightness, projection distance, and projection angle, so as to ensure the quality and visibility of the projection effect; In this embodiment, the projection area is an area that satisfies preset projection conditions and is determined based on data analysis, and can be projected. In a classroom, a wall or a screen may be determined as a projection area; In this embodiment, the area to be projected is an area selected from the projectable area for projecting content, for example, a specific area is determined on the selected blackboard as the area to be projected for projecting teaching content; In this embodiment, the projection content: the content in the area to be projected determined according to the type of teaching task and the simulation scene data. The projection content may be a simulation of the experimental process, a teaching video, interactive teaching content, etc.; In this embodiment, projection dimension: a dimensional parameter of a projection determined according to the projection content, for example, a two-dimensional plane projection and a three-dimensional stereoscopic projection; In this embodiment, the projection distance refers to the distance from the projection device to the area to be projected, which affects the clarity and size of the projected content in the projection area. For example, in a classroom, the projection distance may be the distance from the projector to the blackboard, which determines the size and clarity of the projected content on the blackboard. In this embodiment, projection angle refers to the angle of the projection device relative to the area to be projected, which affects the projection position and direction of the projected content in the area; In this embodiment, the projection sub-content refers to decomposing the content into different projection parts for projection according to different parts or characteristics of the projection content. For example, in a teaching system, the projection sub-content may be different chapters of a teaching video or different modules of teaching content, which are projected in different areas respectively. In this embodiment, the environmental change parameter refers to the environmental conditions that may change during the projection process, affecting the projection effect and user experience; In this embodiment, light parameters: light intensity and lighting conditions near the area to be projected, for example, the brightness and direction of the lights in the classroom, which affect the clarity and visibility of the projected content; In this embodiment, interference source parameters: factors that may interfere with the projection effect, including other light sources, interference between projection devices, obstructions, etc., which will affect the clarity and stability of the projected content; In this embodiment, communication parameters: parameters that affect communication and collaboration between projection devices. For example, in a multi-projection device scenario, communication parameters may include communication speed, collaboration mode, data transmission stability, etc. between devices. In this embodiment, interactive items are elements or functions that users can interact with, such as buttons, annotations, trigger actions, etc. Specifically, in the teaching system, interactive items can be clicking a button to view detailed information, dragging an annotation object, etc.; In this embodiment, the interaction parameters are used to describe and control the settings of the interactive project behavior, including trigger conditions, response methods, effects, etc. Specifically, in the teaching system, the interaction parameters can be displaying related content after clicking a button, triggering animation effects after dragging a marked object, etc.; In this embodiment, the preset area is an area pre-set in the area to be projected for displaying specific content or interactive items. For example, in a teaching system, the preset area may be a certain area of ​​the screen for displaying interactive content or prompt information. In this embodiment, interactive prompt information: prompts or instructions provided for specific interactive items or scenarios to help users understand how to interact with the system. Specifically, in the teaching system, the interactive prompt information may be clicking a button to view detailed content, or dragging a marked object for interaction; In this embodiment, the operation prompt manual is a document or guide that provides detailed instructions or instructions to users on system operation and function usage. For example, in a teaching system, the operation prompt manual may be a document that includes system function introduction, operation steps, answers to frequently asked questions, etc.

[0036] The implementation principle and beneficial effects of this embodiment: The present invention uses a series of sub-modules to start with the user's position to select instructions, determine the projection position, projection content, projection equipment and other parameters, perform calibration and projection, and monitor the environmental change parameters in real time during the projection process to achieve adaptive adjustment of the projection parameters, and finally display the interactive content in the area to be projected and provide an operation prompt manual. The present invention can select instructions and projection content according to the position specified by the user to achieve a personalized projection experience. At the same time, by real-time monitoring of environmental change parameters and adjusting projection parameters, the stability and adaptability of the projection effect are guaranteed, and interactive items can be displayed in real time, and interactive prompt information can be generated to improve user participation and learning effects, and then by providing an operation prompt manual, help users better understand and use the system and improve user experience.

[0037] An embodiment of the present invention provides a teaching system based on AR interaction, wherein the interaction module includes: The motion capture submodule is used to capture the user's body movements in real time through a preset motion capture device to generate motion capture data; A limb motion analysis submodule, used for analyzing the motion capture data to generate motion capture analysis data; A trigger determination submodule is used to perform posture analysis on the motion capture analysis data in combination with a preset classification decision algorithm, determine the user's action type and output the posture analysis result; ; in, Represents the i-th posture judgment value, when When , it indicates that the user's action type is the i-th action posture. When , it indicates that the user's action type is not the i-th action posture; represents the exponential function; Representation and The error adjustment factor corresponding to the data point; represents the function used to make classification decisions on the trajectories of the motion capture data; represents the total number of data points in the training data set corresponding to the i-th posture in the preset classification decision algorithm; represents the Langrange multiplier corresponding to the jth data point; Indicates The category labels corresponding to the training samples; Indicates the calculation for training samples and samples to be analyzed in motion capture analysis data The kernel function of the similarity between them; represents the bias term used to adjust the decision boundary; represents the threshold used for posture determination of the i-th posture; An interactive instruction generation submodule is used to verify and compare the gesture analysis result with the adjustable parameters in the simulation scene data at the same time to determine the interactive instruction input by the user; An instruction execution submodule, used to adjust and control the adjustable parameters in the simulation scene data based on the interactive instruction, and output parameter change data; A projection parameter adjustment submodule, used to adjust the projection parameters of the projection device based on the parameter change data, and dynamically adjust the simulation scene data; The effect evaluation submodule is used to obtain user feedback data, evaluate the interaction effect based on preset evaluation indicators, and output the effect analysis results; The system optimization submodule is used to optimize the interactive teaching process based on the effect analysis results and in combination with preset optimization goals.

[0038] In this embodiment, the preset motion capture device is a device that is pre-set to capture the user's body movements, usually a sensor, a camera, etc. For example, a camera-based somatosensory control device can capture the user's hand movements for interaction; In this embodiment, motion capture data refers to raw data obtained from a motion capture device, which generally includes information such as the position and speed of the user's limb movement, for example, the motion trajectory data of the user's hand joints captured by a sensor, etc.; In this embodiment, motion capture analysis data: data obtained after processing and analyzing the motion capture data, used to describe the user's body movement characteristics, for example, extracting specific gesture or action information from the original arm movement data; In this embodiment, a classification decision algorithm is preset: an algorithm model for performing posture analysis and action type determination on motion capture parsing data, for example, using a support vector machine (SVM) algorithm to classify and identify user gesture data; In this embodiment, posture analysis: analyzing and identifying the user's body movement characteristics to determine the user's current posture or movement type, for example, identifying the user's arm extension movement or rotation movement; In this embodiment, the action type is a specific action or posture currently performed by the user determined according to the posture analysis result, for example, the user makes a fist, waves, etc.; In this embodiment, the posture analysis result: the user action type or posture determination result obtained after analyzing the motion capture data according to a preset algorithm, for example, the analysis result of the arm movement data shows that the user is making a hand-raising action; In this embodiment, verification and comparative analysis: comparing and verifying the gesture analysis results with the adjustable parameters in the simulated scene data to determine the interactive instructions input by the user, for example, comparing the relationship between the user gesture recognition results and the position of the object in the virtual scene to confirm whether the user wants to move the object; In this embodiment, parameter change data: data generated after adjusting the adjustable parameters in the simulation scene according to the user interaction instruction, describing the change of the parameters, for example, when the user gestures to a specific area, the light intensity parameter in the simulation scene changes; In this embodiment, feedback data: feedback information of the user on the system interaction process, which may include the user's satisfaction, degree of understanding, etc. For example, teachers and students rate or make suggestions on the teaching content provided by the system; In this embodiment, preset evaluation indicators: indicators set in advance for evaluating interaction effects, which may include accuracy, user participation, etc. Specifically, for example, the accuracy of user action recognition, the user participation in the interaction process, etc. are evaluated; In this embodiment, the effect analysis result is: the conclusion obtained after evaluating the interaction effect based on the user feedback data and the preset evaluation indicators. For example, the system obtains the user's feedback on the teaching system based on the user's rating and participation data analysis; In this embodiment, the preset optimization goals are used to guide the optimization of the interactive teaching process, which may include improving teaching efficiency, enhancing user experience, etc. Specifically, the optimization goals may include improving the accuracy of motion recognition, reducing the difficulty of user operation, etc.

[0039] Implementation principles and beneficial effects of this embodiment: The present invention can realize a personalized teaching process based on AR interaction by coordinating submodules such as motion capture, analysis, posture analysis, feedback and optimization, and can perform real-time interaction and control according to the user's body movements. The system can evaluate the interaction effect through posture analysis results and user feedback data, help improve the teaching process, and automatically adjust the interaction process according to the effect evaluation results and optimization goals to achieve intelligent optimization of the teaching process.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A teaching system based on AR interaction, characterized in that: include: The data acquisition module is used to use a preset scanning device to acquire the environmental data of the real scene in real time, output the real-time scene data, and at the same time, acquire the type of teaching task and the simulation scene data under the corresponding type; A data conversion module is used to perform data conversion on the real-time scene data using a preset scene construction algorithm, output scene conversion data, and update the scene database, and at the same time, match the scene conversion data with the simulation scene data; A virtual reality combination module is used to project the simulated scene data that meets the preset matching conditions into the real scene corresponding to the scene conversion data; The interactive module is used to obtain the user's interactive instructions in real time, and adjust and control the adjustable parameters in the simulation scene data based on the interactive instructions, and at the same time, dynamically adjust the corresponding simulation scene data in combination with the variable parameters of the adjustable parameters.

2. The teaching system based on AR interaction according to claim 1, characterized in that: The data acquisition module comprises: A real scene scanning submodule, used to use the preset scanning device to perform a three-dimensional scan of the real scene environment around the device in real time, and to construct a scene based on the scanned data to obtain real-time scene data; The virtual scene acquisition submodule is used to acquire the type of teaching task and acquire the corresponding simulation scene data under the type of teaching task in the scene database.

3. The teaching system based on AR interaction according to claim 2, characterized in that: The real scene scanning submodule comprises: A first solution building unit, configured to scan a real scene around the device at different positions using the same preset scanning device, and output a first scanning solution; A second scheme constructing unit, configured to construct a scanning device array according to at least two preset scanning devices at multiple different positions, and output a second scanning scheme; The data scanning unit is used to obtain three-dimensional data of the real environment around the device in real time based on the first scanning scheme and the second scanning scheme through a preset scanning device, and to perform scene construction to obtain real-time scene data.

4. The teaching system based on AR interaction according to claim 3 is characterized in that: The data scanning unit comprises: A scheme determination subunit is used to calculate the complexity of the target real scene according to a preset algorithm, and compare and analyze it with a preset threshold condition. When the scene complexity is less than a first preset complexity, a first scanning scheme is adopted, otherwise, a second scanning scheme is adopted; The data scanning subunit is used to determine the scanning scheme adopted by the subunit based on the scheme, and to obtain the three-dimensional data of the real environment around the device in real time through a preset scanning device, and to construct the scene to obtain real-time scene data.

5. The teaching system based on AR interaction according to claim 1, characterized in that: The data conversion module comprises: A scene conversion submodule, used to perform data conversion on the real-time scene data according to the preset scene construction algorithm, and output scene conversion data; A data updating submodule, used for comparing and analyzing the scene conversion data with the constructed scene data in the scene database, and determining whether to update the scene conversion data to the scene database based on the comparison and analysis result; The data matching submodule is used to perform three-dimensional matching between the scene conversion data and the simulation scene data, and determine the projection parameters of the simulation scene data based on the matching result.

6. The teaching system based on AR interaction according to claim 5, characterized in that: The data updating submodule comprises: A data analysis unit, used to analyze the similarity and difference between the scene conversion data and the constructed scene data in the scene database; The update determination unit is used to compare and analyze the similarity and difference with the preset data update threshold condition, upload and update the scene conversion data that meets the preset data update threshold condition to the scene database, otherwise, the scene conversion data is not updated to the scene database.

7. The teaching system based on AR interaction according to claim 5, characterized in that: The virtual reality combining module comprises: A projection position selection submodule, used to determine a position area for projection by a projection device in a real scene based on the real-time scene data and in combination with a user's position selection instruction; wherein, the scene conversion data is analyzed, and an area where the analysis result satisfies a preset projection condition is output as a projection-enabled area; Obtaining a location selection instruction from a user, and selecting an area to be projected from each of the projection-enabled areas; A content acquisition submodule, used to determine the projection content and projection dimension in the to-be-projected area based on the type of teaching task and the simulation scene data under the corresponding type; A device selection submodule, used to determine the number of projection devices to be called based on the projection content and projection dimension, and determine the projection distance, projection angle and projection sub-content of each projection device; The calibration submodule is used to calibrate and adjust each projection device before projection, and determine the projection parameters of each projection device; A projection submodule, configured to project the projection sub-content of the corresponding device into the to-be-projected area based on the projection of each projection device; An adaptive submodule, used to obtain environmental change parameters in the projection process in real time, and to adaptively adjust the projection parameters of each projection device in real time based on the environmental change data, wherein the environmental change parameters include light parameters near the projection area, interference source parameters, and communication parameters of each projection device; The function display submodule is used to obtain the interactive items and corresponding interactive parameters corresponding to the current real-time display screen in the projection content, and display them in real time in a preset area in the area to be projected, and at the same time, mark the interactive items with information and generate interactive prompt information; The self-service submodule is used to provide the user with an operation prompt manual corresponding to the current real-time display screen.

8. The teaching system based on AR interaction according to claim 1, characterized in that: The interactive module comprises: The motion capture submodule is used to capture the user's body movements in real time through a preset motion capture device to generate motion capture data; A limb motion analysis submodule, used for analyzing the motion capture data to generate motion capture analysis data; A trigger determination submodule is used to perform posture analysis on the motion capture analysis data in combination with a preset classification decision algorithm, determine the user's action type and output the posture analysis result; ； in, represents the i-th posture judgment value, when When , it indicates that the user's action type is the i-th action posture. When , it indicates that the user's action type is not the i-th action posture; represents the exponential function; Representation and The error adjustment factor corresponding to the data point; represents the function used to make classification decisions on the trajectories of the motion capture data; represents the total number of data points in the training data set corresponding to the i-th posture in the preset classification decision algorithm; represents the Langrange multiplier corresponding to the jth data point; Indicates The category labels corresponding to the training samples; Indicates the calculation for training samples and samples to be analyzed in the motion capture analysis data The kernel function of the similarity between them; represents the bias term used to adjust the decision boundary; represents the threshold used for posture determination of the i-th posture; An interactive instruction generation submodule is used to verify and compare the gesture analysis result with the adjustable parameters in the simulation scene data at the same time to determine the interactive instruction input by the user; An instruction execution submodule, used to adjust and control the adjustable parameters in the simulation scene data based on the interactive instruction, and output parameter change data; A projection parameter adjustment submodule, used to adjust the projection parameters of the projection device based on the parameter change data, and dynamically adjust the simulation scene data; The effect evaluation submodule is used to obtain user feedback data, evaluate the interaction effect based on preset evaluation indicators, and output the effect analysis results; The system optimization submodule is used to optimize the interactive teaching process based on the effect analysis results and in combination with preset optimization goals.

Images