AR-assisted physical phenomenon real-time identification and interpretation system

Abstract

The invention relates to the technical field of augmented reality (AR) and education, in particular to an AR-assisted physical phenomenon real-time identification and interpretation system, which comprises the following modules: a data acquisition module, a physical phenomenon identification module, an interpretation content generation module, an AR rendering and presentation module and a user interaction and personalization module. Physical phenomena in real life are converted into visual AR teaching cases, physical principles are presented in the form of dynamic animation and formula derivation, students are helped to understand abstract physical knowledge more easily, and the learning efficiency and effect are improved.

Description

Technical Field

[0001] The present invention relates to the fields of augmented reality (AR) technology and educational technology, and in particular to an AR-assisted real-time recognition and interpretation system for physical phenomena. Background Art

[0002] In traditional physics teaching, the explanation of physical phenomena often relies on pictures and text descriptions in textbooks, or simple experimental demonstrations. For some abstract and complex physical phenomena, such as microscopic molecular motion and macroscopic celestial mechanics, it is difficult for students to intuitively understand their principles and laws. At the same time, there are a large number of physical phenomena in life, but due to the lack of effective identification and interpretation methods, these rich teaching resources are not fully utilized. Existing AR educational applications have deficiencies in the real-time and accuracy of physical phenomenon identification, as well as the depth and visualization of the explanation content, and cannot meet the needs of high-quality physics teaching. To this end, an AR-assisted real-time identification and interpretation system for physical phenomena is proposed. Summary of the Invention

[0003] In view of this, the present invention provides an AR-assisted real-time recognition and interpretation system for physical phenomena to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0004] The technical solution of the present invention is implemented as follows: an AR-assisted real-time recognition and interpretation system for physical phenomena includes the following modules: a data acquisition module, a physical phenomenon recognition module, an interpretation content generation module, an AR rendering and presentation module, and a user interaction and personalization module.

[0005] Further preferably, the data acquisition module includes a visual acquisition unit and a sensor acquisition unit;

[0006] Visual acquisition unit: A mobile phone camera or a built-in camera in AR glasses captures images of the real environment at a frame rate of at least 30 frames per second, capturing the visual characteristics of physical phenomena, such as water flow patterns and object motion trajectories.

[0007] Sensor acquisition unit: Integrates sensors such as accelerometers, gyroscopes, and magnetometers to collect the device's motion state, spatial position, and direction information in real time, assisting in determining the scene and dynamic changes of physical phenomena. For example, accelerometer data can be used to determine the acceleration or deceleration of an object.

[0008] Further preferably, the physical phenomenon recognition module includes an image preprocessing submodule, a feature extraction submodule, a multimodal fusion submodule and a phenomenon recognition submodule;

[0009] Image preprocessing submodule: performs preprocessing operations such as noise reduction, enhancement, and edge detection on the collected images to improve image quality and highlight the key features of physical phenomena;

[0010] Feature extraction submodule: This module uses convolutional neural networks (CNNs) in deep learning to extract physical phenomenon features in images, such as the fringe features of light wave interference and the fluid motion features of thermal convection.

[0011] Multimodal fusion submodule: This module fuses visual features with sensor data to construct a feature vector containing multi-dimensional information such as space, time, and motion, thereby improving the accuracy and robustness of physical phenomenon recognition.

[0012] Phenomenon recognition submodule: Based on the trained recognition model (such as ResNet, YOLO and other improved models), the fused features are classified and recognized to determine the type of physical phenomenon existing in the real environment.

[0013] Further preferably, the explanation content generation module includes a knowledge graph construction submodule, a content matching submodule, an animation generation submodule and a formula derivation generation submodule;

[0014] Knowledge graph construction submodule: Integrates the principles, formulas, experimental cases, and other contents of physics to construct a physics knowledge graph, clarify the relationship between various knowledge points, and provide knowledge support for the generation of explanatory content;

[0015] Content matching submodule: Based on the identified physical phenomenon type, the corresponding principle knowledge, relevant formulas and typical cases are retrieved in the knowledge graph to select the appropriate explanation content;

[0016] Animation generation submodule: Utilizes computer graphics and animation production technology to transform physical principles into dynamic principle animations. Through keyframe settings and physical engine simulation, it intuitively displays the occurrence process and internal mechanisms of physical phenomena.

[0017] Formula derivation generation submodule: Based on the screened formulas, a step-by-step derivation animation is generated in accordance with the logical derivation order, clearly presenting the evolution process and application scenarios of the physical formulas.

[0018] Further preferably, the AR rendering and presentation module includes a space positioning submodule, a virtual object rendering submodule and a virtual-reality fusion submodule;

[0019] Spatial positioning submodule: Through SLAM (Simultaneous Localization and Mapping) technology, it builds a three-dimensional map of the real environment in real time, determines the position and posture of the device in the environment, and provides spatial coordinates for the accurate overlay of virtual information;

[0020] Virtual object rendering submodule: This module converts the generated principle animation and formula derivation content into a virtual object model, renders it using an AR rendering engine (such as ARKit or ARCore), and sets the virtual object's color, material, lighting effects, and other properties to enhance visual realism.

[0021] Virtual-reality fusion submodule: Based on the spatial positioning results, the rendered virtual objects are accurately superimposed on the positions corresponding to the physical phenomena in the real scene, achieving seamless fusion of virtual information and the real scene. The position and posture of the virtual objects are updated in real time to maintain the consistency and coherence of the virtual-reality fusion.

[0022] Further preferably, the user interaction and personalization module includes an interactive input submodule, a learning status monitoring submodule and a personalization adjustment submodule;

[0023] Interactive Input Submodule: This module supports user interaction with the system through various methods, including touch screen, voice commands, and gestures. For example, users can ask questions through voice to obtain more detailed explanations of physical principles; and use gestures to zoom in, out, or rotate virtual objects to observe principle animations from different angles.

[0024] Learning status monitoring submodule: records user learning behavior data, such as learning time, interactive operations, incorrect answers, etc., and analyzes the user's learning progress, knowledge mastery, and operating habits;

[0025] Personalized adjustment submodule: Based on the learning status monitoring results, the difficulty level, presentation speed and interaction method of the explanation content are dynamically adjusted to provide users with a personalized learning experience. For example, for users with weak foundations, the difficulty of the explanation content is reduced, and examples and explanation details are added.

[0026] The embodiment of the present invention adopts the above technical solution, which has the following advantages:

[0027] 1. This invention transforms real-life physical phenomena into intuitive AR teaching cases, presenting physical principles in the form of dynamic animations and formula derivations, helping students understand abstract physical knowledge more easily and improving learning efficiency and effectiveness.

[0028] 2. The present invention makes full use of physical phenomena in life scenarios as teaching materials, expands the resource sources of physics teaching, makes physics learning no longer limited to classrooms and laboratories, and stimulates students' interest in learning physics.

[0029] 3. The virtual-real fusion effect and personalized learning adaptation achieved by the AR technology of the present invention bring users an immersive and interactive learning experience, meet the learning needs of different users, and enhance the fun and participation of learning.

[0030] 4. The present invention deeply integrates AR technology with physics teaching, providing a new teaching model and method for the field of education, which helps to promote the innovative development of educational technology and promote the modernization of education.

[0031] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0034] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and description are to be considered as illustrative in nature and not restrictive.

[0035] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, an embodiment of the present invention provides an AR-assisted real-time recognition and interpretation system for physical phenomena, including the following modules: data acquisition module, physical phenomenon recognition module, interpretation content generation module, AR rendering and presentation module, and user interaction and personalization module.

[0037] In one embodiment, the data acquisition module includes a visual acquisition unit and a sensor acquisition unit;

[0038] Visual acquisition unit: A mobile phone camera or a built-in camera in AR glasses captures images of the real environment at a frame rate of at least 30 frames per second, capturing the visual characteristics of physical phenomena, such as water flow patterns and object motion trajectories.

[0039] Sensor acquisition unit: Integrates sensors such as accelerometers, gyroscopes, and magnetometers to collect the device's motion state, spatial position, and direction information in real time, assisting in determining the scene and dynamic changes of physical phenomena. For example, accelerometer data can be used to determine the acceleration or deceleration of an object.

[0040] In one embodiment, the physical phenomenon recognition module includes an image preprocessing submodule, a feature extraction submodule, a multimodal fusion submodule, and a phenomenon recognition submodule;

[0041] Image preprocessing submodule: performs preprocessing operations such as noise reduction, enhancement, and edge detection on the collected images to improve image quality and highlight the key features of physical phenomena;

[0042] Feature extraction submodule: This module uses convolutional neural networks (CNNs) in deep learning to extract physical phenomenon features in images, such as the fringe features of light wave interference and the fluid motion features of thermal convection.

[0043] Multimodal fusion submodule: This module fuses visual features with sensor data to construct a feature vector containing multi-dimensional information such as space, time, and motion, thereby improving the accuracy and robustness of physical phenomenon recognition.

[0044] Phenomenon recognition submodule: Based on the trained recognition model (such as ResNet, YOLO and other improved models), the fused features are classified and recognized to determine the type of physical phenomenon existing in the real environment.

[0045] In one embodiment, the explanation content generation module includes a knowledge graph construction submodule, a content matching submodule, an animation generation submodule, and a formula derivation generation submodule;

[0046] Knowledge graph construction submodule: Integrates the principles, formulas, experimental cases, and other contents of physics to construct a physics knowledge graph, clarify the relationship between various knowledge points, and provide knowledge support for the generation of explanatory content;

[0047] Content matching submodule: Based on the identified physical phenomenon type, the corresponding principle knowledge, relevant formulas and typical cases are retrieved in the knowledge graph to select the appropriate explanation content;

[0048] Animation generation submodule: Utilizes computer graphics and animation production technology to transform physical principles into dynamic principle animations. Through keyframe settings and physical engine simulation, it intuitively displays the occurrence process and internal mechanisms of physical phenomena.

[0049] Formula derivation generation submodule: Based on the screened formulas, a step-by-step derivation animation is generated in accordance with the logical derivation order, clearly presenting the evolution process and application scenarios of the physical formulas.

[0050] In one embodiment, the AR rendering and presentation module includes a spatial positioning submodule, a virtual object rendering submodule, and a virtual-reality fusion submodule;

[0051] Spatial positioning submodule: Through SLAM (Simultaneous Localization and Mapping) technology, it builds a three-dimensional map of the real environment in real time, determines the position and posture of the device in the environment, and provides spatial coordinates for the accurate overlay of virtual information;

[0052] Virtual object rendering submodule: Converts the generated principle animation and formula derivation content into a virtual object model, uses an AR rendering engine (such as ARKit, ARCore) for rendering, and sets the virtual object's color, material, lighting effects and other properties to improve visual realism;

[0053] Virtual-reality fusion submodule: Based on the spatial positioning results, the rendered virtual objects are accurately superimposed on the positions corresponding to the physical phenomena in the real scene, achieving seamless fusion of virtual information and the real scene. The position and posture of the virtual objects are updated in real time to maintain the consistency and coherence of the virtual-reality fusion.

[0054] In one embodiment, the user interaction and personalization module includes an interaction input submodule, a learning status monitoring submodule, and a personalization adjustment submodule;

[0055] Interactive Input Submodule: This module supports user interaction with the system through various methods, including touch screen, voice commands, and gestures. For example, users can ask questions through voice to obtain more detailed explanations of physical principles; and use gestures to zoom in, out, or rotate virtual objects to observe principle animations from different angles.

[0056] Learning status monitoring submodule: records user learning behavior data, such as learning time, interactive operations, incorrect answers, etc., and analyzes the user's learning progress, knowledge mastery, and operating habits;

[0057] Personalized adjustment submodule: Based on the learning status monitoring results, the difficulty level, presentation speed and interaction method of the explanation content are dynamically adjusted to provide users with a personalized learning experience. For example, for users with weak foundations, the difficulty of the explanation content is reduced, and examples and explanation details are added.

[0058] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various modifications and substitutions within the technical scope disclosed in the present invention, and such modifications and substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. AR-assisted real-time recognition and interpretation system for physical phenomena, characterized by: It includes the following modules: data acquisition module, physical phenomenon recognition module, explanation content generation module, AR rendering and presentation module, and user interaction and personalization module.

2. The AR-assisted real-time recognition and interpretation system for physical phenomena according to claim 1, characterized in that: The data acquisition module includes a visual acquisition unit and a sensor acquisition unit; Visual acquisition unit: A mobile phone camera or a built-in camera in AR glasses captures images of the real environment at a frame rate of at least 30 frames per second, capturing the visual characteristics of physical phenomena, such as water flow patterns and object motion trajectories. Sensor acquisition unit: Integrates sensors such as accelerometers, gyroscopes, and magnetometers to collect the device's motion state, spatial position, and direction information in real time, assisting in determining the scene and dynamic changes of physical phenomena. For example, accelerometer data can be used to determine the acceleration or deceleration of an object.

3. The AR-assisted real-time recognition and interpretation system for physical phenomena according to claim 1, characterized in that: The physical phenomenon recognition module includes an image preprocessing submodule, a feature extraction submodule, a multimodal fusion submodule and a phenomenon recognition submodule; Image preprocessing submodule: performs preprocessing operations such as noise reduction, enhancement, and edge detection on the collected images to improve image quality and highlight the key features of physical phenomena; Feature extraction submodule: This module uses convolutional neural networks (CNNs) in deep learning to extract physical phenomenon features in images, such as the fringe features of light wave interference and the fluid motion features of thermal convection. Multimodal fusion submodule: This module fuses visual features with sensor data to construct a feature vector containing multi-dimensional information such as space, time, and motion, thereby improving the accuracy and robustness of physical phenomenon recognition. Phenomenon recognition submodule: Based on the trained recognition model (such as ResNet, YOLO and other improved models), the fused features are classified and recognized to determine the type of physical phenomenon existing in the real environment.

4. The AR-assisted real-time recognition and interpretation system for physical phenomena according to claim 1, characterized in that: The explanation content generation module includes a knowledge graph construction submodule, a content matching submodule, an animation generation submodule and a formula derivation generation submodule; Knowledge graph construction submodule: Integrates the principles, formulas, experimental cases, and other contents of physics to construct a physics knowledge graph, clarify the relationship between various knowledge points, and provide knowledge support for the generation of explanatory content; Content matching submodule: Based on the identified physical phenomenon type, the corresponding principle knowledge, relevant formulas and typical cases are retrieved in the knowledge graph to select the appropriate explanation content; Animation generation submodule: Utilizes computer graphics and animation production technology to transform physical principles into dynamic principle animations. Through keyframe settings and physical engine simulation, it intuitively displays the occurrence process and internal mechanisms of physical phenomena. Formula derivation generation submodule: Based on the screened formulas, a step-by-step derivation animation is generated in accordance with the logical derivation order, clearly presenting the evolution process and application scenarios of the physical formulas.

5. The AR-assisted real-time recognition and interpretation system for physical phenomena according to claim 1, characterized in that: The AR rendering and presentation module includes a spatial positioning submodule, a virtual object rendering submodule and a virtual-reality fusion submodule; Spatial positioning submodule: Through SLAM (Simultaneous Localization and Mapping) technology, it builds a three-dimensional map of the real environment in real time, determines the position and posture of the device in the environment, and provides spatial coordinates for the accurate overlay of virtual information; Virtual object rendering submodule: Converts the generated principle animation and formula derivation content into a virtual object model, uses an AR rendering engine (such as ARKit, ARCore) for rendering, and sets the virtual object's color, material, lighting effects and other properties to improve visual realism; Virtual-reality fusion submodule: Based on the spatial positioning results, the rendered virtual objects are accurately superimposed on the positions corresponding to the physical phenomena in the real scene, achieving seamless fusion of virtual information and the real scene. The position and posture of the virtual objects are updated in real time to maintain the consistency and coherence of the virtual-reality fusion.

6. The AR-assisted real-time recognition and interpretation system for physical phenomena according to claim 1, characterized in that: The user interaction and personalization module includes an interactive input submodule, a learning status monitoring submodule, and a personalization adjustment submodule; Interactive Input Submodule: This module supports user interaction with the system through various methods, including touch screen, voice commands, and gestures. For example, users can ask questions through voice to obtain more detailed explanations of physical principles; and use gestures to zoom in, out, or rotate virtual objects to observe principle animations from different angles. Learning status monitoring submodule: records user learning behavior data, such as learning time, interactive operations, incorrect answers, etc., and analyzes the user's learning progress, knowledge mastery, and operating habits; Personalized adjustment submodule: Based on the learning status monitoring results, the difficulty level, presentation speed and interaction method of the explanation content are dynamically adjusted to provide users with a personalized learning experience. For example, for users with weak foundations, the difficulty of the explanation content is reduced, and examples and explanation details are added.

Images