An interactive globe integrating intelligent tutoring and internet of things functions and method

Abstract

The application provides an interactive globe integrating intelligent tutoring and Internet of Things functions and a method thereof.The interactive globe integrating intelligent tutoring and Internet of Things functions comprises a sphere, a base and an intelligent host module, integrates hardware such as a touch screen, a homework tutoring camera, a miniature projection, a microphone array, a loudspeaker and the like, carries a double-layer permission control system and a multi-modal interaction system, accesses a cloud AI model, and realizes integration of geographical education, AI tutoring, voice interaction, screen projection printing and smart home control.The working method distinguishes between the modes of minors and adults through permission differentiation, ensures use safety, improves operation friendliness through multi-modal interaction, solves the problems of single function, inconvenient interaction, weak expansibility and no permission management of traditional intelligent globes, and is suitable for use in multiple scenes such as teaching, learning, entertainment and control in families.

Description

Technical Field

[0001] This application relates to the field of smart globe technology, and more specifically, to an interactive globe and method that integrates smart tutoring and Internet of Things (IoT) functions. Background Technology

[0002] Traditional globes are limited in function, primarily used for statically displaying geographical and administrative information, lacking interactivity and expandability. Existing smart educational devices, such as reading pens and learning machines, while offering some tutoring functions, have content and presentation formats that are detached from physical globe models, failing to provide an intuitive and immersive geospatial learning experience.

[0003] In recent years, some "smart globes" with simple LED lights or AR recognition functions have appeared on the market, but their core functions are still limited to information display, and they have the following obvious drawbacks: Low functional integration: It is only a simple combination of "globe + display screen" or "globe + APP", without deep integration of the core functions of modern smart hardware such as homework tutoring, intelligent voice assistant, and home IoT control, and cannot meet the integrated needs of "teaching, learning, entertainment and control" in the home scenario.

[0004] The interaction mode is limited: it mainly relies on mobile apps or simple touch buttons, lacks natural and multimodal interaction (such as integrated interaction that combines voice, motion sensing and touch screen), is not user-friendly for children, and cannot effectively distinguish between adult and child usage permissions, posing a risk of children misoperating.

[0005] Weak expansion and connectivity: They usually lack professional screen projection and printing output interfaces, or have limited connectivity, making it difficult to conveniently serve as a home smart device hub or work in conjunction with other devices (such as projectors and printers) as a terminal for displaying and sharing learning outcomes.

[0006] Usage limitations: It is essentially still a closed learning tool, failing to effectively connect to a wider AI service ecosystem (such as large AI models like Doubao) and smart home ecosystem, and its scalability and intelligence level are limited. Summary of the Invention

[0007] This application aims to address at least one of the technical problems existing in the prior art.

[0008] To this end, this application proposes an interactive globe integrating intelligent tutoring and IoT functions, including a globe sphere, a base, and an intelligent host module disposed within the base; the globe sphere is connected to the base via a pivot and can rotate relative to the base; the base integrates a touch screen, a homework tutoring camera with an adjustable tilt angle, a miniature projector, a speaker, a microphone array, and physical function buttons; the intelligent host module includes a main controller, a storage unit, an AI processing unit, a Wi-Fi / Bluetooth communication module, an audio processing unit, a power management module, sensors, and I / O interfaces; the intelligent host module is configured with a dual-layer permission control system and a multimodal intelligent interaction system, which can realize permission switching between a minor mode and an adult mode, and supports mixed interaction of voice, touch screen, gesture recognition, and physical buttons.

[0009] Preferably, the globe sphere is a traditional printed sphere or a transparent sphere with a built-in LED display screen; the touch display screen is used to display operation menus, detailed geographical information, homework questions, and device status; the homework tutoring camera can be flipped and is set at the front of the base for taking pictures of the homework on the table; the optical engine of the micro projector is built into the base, and its lens opening is located on the side of the base for projecting images to the outside; the sensors include an ambient light sensor and a sphere attitude sensor, which are used to automatically adjust the screen brightness and detect the sphere rotation angle, respectively.

[0010] Preferably, the dual-level access control system achieves mode switching through password, fingerprint, or facial recognition; functions are restricted in the underage mode, while all functions are open in the adult mode. The function sets, AI service access permissions, screen projection permissions, and system setting permissions of the two modes are independent of each other, and mode switching requires higher-level authorization.

[0011] Preferably, the multimodal intelligent interaction system integrates far-field voice recognition, touch screen operation, gesture recognition and physical buttons, and sets up a unified interaction logic and priority processing mechanism.

[0012] Preferably, the smart host module also integrates a screen projection module, a printing service module, and an IoT control module, supporting DLNA / Miracast screen projection, wireless printing, and smart home device control.

[0013] Preferably, the AI ​​processing unit integrates an NPU and connects to a large-scale AI model in the cloud for local speech recognition, accelerated question image recognition, and cloud-based intelligent question answering and problem-solving tutoring.

[0014] On the other hand, this application also provides a method for operating an interactive globe that integrates intelligent tutoring and Internet of Things (IoT) functions, including the following steps: S1. Startup and Authorization Verification: The device is started by touch or voice wake-up. After startup, it enters the authorization verification process. The user completes the verification by password or biometric identification. The system enters the underage mode or adult mode according to the permission level. S2. In the corresponding mode, perform geographic learning, AI voice Q&A, homework tutoring, screen projection / printing or IoT control operations through multimodal interaction.

[0015] Preferred options include: In the underage mode, only basic geography learning, homework tutoring, children's media playback, and filtered AI Q&A are available; authorized screen mirroring is supported only; the network is limited to educational servers; and system-level settings are prohibited. In the adult mode, the full AI assistant, full-featured screen mirroring, full media resources, full IoT control, and all system settings are available.

[0016] Preferably, the homework tutoring process is as follows: turn on the homework tutoring function, point the camera at the question and take a picture, and after OCR recognition and AI problem-solving model analysis, the solution ideas are displayed step by step on the screen, and the globe can be linked to locate and highlight relevant geographical areas.

[0017] Preferably, content can be projected to external display devices via Wi-Fi / Bluetooth, learning materials can be printed wirelessly, and smart home appliances can be controlled via voice commands through an IoT protocol module.

[0018] An interactive globe integrating intelligent tutoring and Internet of Things (IoT) functions according to an embodiment of this application has the following advantages: 1. Achieving high functional integration and scene fusion: This device integrates a physical globe, a miniature projector, a homework tutoring camera, a high-quality speaker, an AI voice module, a wireless communication module, and a touch screen control into one device. It creatively integrates multiple independent scenarios such as geography education, after-school homework tutoring, intelligent voice interaction, home entertainment (music / podcast), and IoT device control into a single device, which greatly saves space and improves the overall effectiveness of the device and the user experience. 2. Provides natural and secure multimodal interaction: By combining touchscreen, far-field voice recognition, and physical knobs / buttons, it offers a more intuitive interaction method. In particular, the innovative two-layer password permission system can strictly distinguish between "minor mode" and "adult mode," protecting children's safety while granting adult users full access to all functions (such as complete IoT control and screen mirroring management), solving the permission management problem when sharing devices.

[0019] 3. Enhanced scalability and ecosystem integration: With built-in screen projection functions (such as DLNA and Miracast), Bluetooth / Wi-Fi connectivity, and a printing service module, this device can easily project learning content onto a large screen, connect Bluetooth headphones / speakers, or directly print learning reports. More importantly, by integrating with large AI models like Doubao, it possesses powerful natural language understanding, knowledge-based question answering, and creative content generation capabilities, surpassing traditional globes with pre-installed content. 4. Enhance learning efficiency and engagement: The homework tutoring camera, combined with AI algorithms, enables problem recognition via image, intelligent grading, and recommended explanations. Music playback and audio stories create a conducive learning atmosphere. This multi-sensory learning mode, combining visual (globe / projection), auditory (explanation / music), and tactile (touch interaction), significantly improves the immersion and efficiency of learning. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a block diagram of a smart host module according to an embodiment of this application; Figure 2 This is a block diagram of a two-layer access control system according to an embodiment of this application; Figure 3 This is a block diagram of a multimodal intelligent interaction system according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] On the one hand, this application proposes an interactive globe integrating intelligent tutoring and Internet of Things functions, including a globe body, a base, and an intelligent host module disposed in the base.

[0024] The globe is connected to the base via a pivot and can rotate relative to the base, either manually or electrically.

[0025] Furthermore, the base integrates a touch screen, a flip-up and adjustable homework guidance camera, a mini projector, a speaker, a microphone array, and physical function buttons (such as volume, power, and mode switching buttons).

[0026] The intelligent host module includes a main controller, storage unit, AI processing unit, Wi-Fi / Bluetooth communication module, audio processing unit, power management module, sensors, and I / O interfaces. The intelligent host module is equipped with a dual-layer permission control system and a multimodal intelligent interaction system, which can realize permission switching between underage mode and adult mode, and supports mixed interaction of voice, touch screen, gesture recognition, and physical buttons.

[0027] Specifically, the globe's sphere is either a traditional printed sphere or a transparent sphere, with a built-in LED display screen. The touch screen displays the operation menu, detailed geographical information, homework questions, and equipment status. The homework guidance camera can be flipped and mounted on the front of the base to photograph the homework on the desktop. The optical engine of the mini projector is built into the base, and its lens opening is located on the side of the base to project images to the outside. The sensors include an ambient light sensor and a sphere attitude sensor, which are used to automatically adjust the screen brightness and detect the sphere's rotation angle, respectively.

[0028] Furthermore, the main controller in the intelligent host module adopts a high-performance ARM processor and runs a customized operating system; the storage unit is used to store map data, teaching resources, and user data; the AI ​​processing unit can integrate an NPU for local speech recognition, image recognition (homework questions) acceleration, or as a coprocessor for communication with cloud AI services (such as Doubao); the Wi-Fi / Bluetooth communication module is used to connect to the Internet, home LAN, Bluetooth headsets / speakers, printers, etc.; the audio processing unit includes an audio codec and power amplifier for driving speakers and processing microphone input; the power management module is used to support AC adapter power supply and built-in lithium battery charging management to achieve charging and uninterrupted use; sensors may include an ambient light sensor (for automatically adjusting screen brightness) and an attitude sensor (for detecting the rotation angle of a sphere); the I / O interface includes a USB interface (for data transmission and connecting peripherals) and an HDMI input / output port (optional, for expanding screen projection function).

[0029] In a specific embodiment of this application, the dual-level access control system achieves mode switching through password, fingerprint, or facial recognition; functions are restricted in the underage mode, while all functions are open in the adult mode. The function sets, AI service access permissions, screen projection permissions, and system setting permissions of the two modes are independent of each other, and mode switching requires higher-level authorization.

[0030] The multimodal intelligent interaction system integrates far-field voice recognition, touch screen operation, gesture recognition, and physical buttons, and sets up a unified interaction logic and priority processing mechanism.

[0031] Furthermore, the smart host module also integrates a screen projection module, a printing service module, and an IoT control module, supporting DLNA / Miracast screen projection, wireless printing, and smart home device control.

[0032] Furthermore, the AI ​​processing unit integrates an NPU and connects to a large-scale AI model in the cloud for local speech recognition, question image recognition acceleration, and cloud-based intelligent question answering and problem-solving tutoring.

[0033] On the other hand, this application also provides a method for operating an interactive globe that integrates intelligent tutoring and Internet of Things (IoT) functions, including the following steps: S1. Startup and Authorization Verification: The device is started by touch or voice wake-up. After startup, it enters the authorization verification process. The user completes the verification by password or biometric identification. The system enters the underage mode or adult mode according to the permission level. S2. In the corresponding mode, perform geographic learning, AI voice Q&A, homework tutoring, screen projection / printing or IoT control operations through multimodal interaction.

[0034] Specifically, startup and permission verification: The user wakes up the device by touching the switch or by voice; after the device starts up, it first enters the permission verification interface; the user enters the password (or performs biometric identification); after the system verifies the password, the user enters the first level (minor mode) or the second level (adult mode) according to the password level.

[0035] The differences in mode functionality include a minor mode and an adult mode.

[0036] Minor Mode: Functionality is limited, focusing on learning.

[0037] Accessible content includes: basic geography knowledge, in-school homework tutoring, nursery rhymes / story playback, and limited AI Q&A (filtering unsafe content); the screen mirroring function can only receive authorized screen mirroring from parents' devices and cannot actively search for or mirror to external devices; network connection is limited to educational content servers. System-level settings (such as connecting to new Wi-Fi networks or Bluetooth pairing) are not available.

[0038] Adult Mode: All features enabled.

[0039] Accessible scope: Full version of AI assistant (e.g., Doubao), full-featured screen mirroring (can be used as a transmitter or receiver), all music / podcast resources, complete IoT control functions (control smart home appliances through built-in Tmall Genie or similar services), and all system settings.

[0040] "Use when connecting to screen mirroring" specifically means that in this mode, the device can act as the main control terminal to project its own screen content or globe presentation content to the TV / projector.

[0041] The core functional flow of the interactive globe operating method integrating intelligent guidance and Internet of Things (IoT) functions described in this application is as follows: AI voice interaction: After the user says the wake word (such as "XX classmate"), the voice is picked up by the microphone array, processed by noise reduction and echo cancellation, and the local speech recognition module or cloud service converts the speech into text. The text request is sent to the integrated Doubao AI service backend, and after receiving the intelligent reply, it is played in the form of text-to-speech (TTS) and may display auxiliary information on the touch screen.

[0042] Homework Tutoring: Activate this function, point the camera at the homework question, and the user takes a picture. The image is then processed by local or cloud-based OCR to recognize the text of the question. An AI problem-solving model is then used for analysis, and the solution process and answer are displayed step-by-step on the screen. (It can be linked to a globe; for example, if a math problem involves "time zone," the relevant region will be automatically located and highlighted.)

[0043] Screen mirroring and printing: Screen casting: In adult mode, users can choose "screen mirroring" or "media casting" to project the content (or the video being played) on the globe touchscreen to a smart TV / projector on the same network via Wi-FiDirect or DLNA protocol. Printing: The device discovers and connects to wireless printers via Wi-Fi or Bluetooth. Users can send print commands for content such as learning reports, maps, and knowledge cards.

[0044] IoT control: In adult mode, users can control devices by sending voice commands (such as "turn on the living room lights") via the device's built-in IoT protocol module (such as connecting to the Tmall Genie ecosystem).

[0045] In another embodiment of this application, alternatives to the projection module include: using a more advanced laser projector or DLP optical engine to reduce size, increase brightness and lifespan; or eliminating the built-in projector and replacing it with an external projector connected wirelessly to reduce device complexity and cost.

[0046] AI service alternatives: The core AI capabilities can be replaced by other mainstream domestic models, such as Wenxin Yiyan and Tongyi Qianwen, instead of integrating "Doubao", or by using a lightweight model deployed locally.

[0047] Alternative access verification methods: The dual password can be replaced with "Parents' mobile app authorization to unlock adult mode" or "Children's face recognition automatically enters minor mode, and adults' face recognition enters adult mode".

[0048] Structural alternatives: The globe body can be designed to be detachable, and the base can be used independently as a smart speaker, or a "floating globe" design can be adopted to enhance the sense of technology.

[0049] This application also provides a typical use case, as follows: Children's geography learning: When a child rotates the globe, the touchscreen displays information about the country they are pointing to. When a child asks, "Doubao, tell me about the Egyptian pyramids," the device responds with a voice message and projects an image of the pyramids onto the wall.

[0050] Homework tutoring: When a child enters the minor mode, the camera is used to take a picture of a math word problem. After the device recognizes the image, the solution process is animated and displayed on the screen step by step.

[0051] Family entertainment and control: Parents enter a password to access adult mode and say, "Play Jay Chou's songs." The music starts playing; they say, "Project this world climate map onto the living room TV." The TV displays it simultaneously; they say, "Turn off the bedroom air conditioner." The air conditioner turns off immediately.

[0052] The above embodiments are only used to illustrate specific implementations of the present invention and are not limited thereto. For those skilled in the art, various similar modifications and transformations can be made based on the concept of the present invention, and these modifications and transformations should all be considered within the scope of protection of the present invention.

Claims

1. An interactive globe integrating intelligent tutoring and Internet of Things (IoT) functions, characterized in that, include: The globe, the base, and the intelligent host module located within the base; The globe sphere is connected to the base via a pivot and can rotate relative to the base; The base integrates a touch screen, a flip-up and adjustable homework guidance camera, a mini projector, a speaker, a microphone array, and physical function buttons. The intelligent host module includes a main controller, a storage unit, an AI processing unit, a Wi-Fi / Bluetooth communication module, an audio processing unit, a power management module, sensors, and I / O interfaces; The intelligent host module is equipped with a dual-layer permission control system and a multimodal intelligent interaction system, which can realize permission switching between underage mode and adult mode, and supports mixed interaction of voice, touch screen, gesture recognition and physical buttons.

2. An interactive globe integrating intelligent tutoring and Internet of Things functions as described in claim 1, characterized in that, The globe sphere is a traditional printed sphere or a transparent sphere with a built-in LED display screen; The touch screen is used to display the operation menu, detailed geographic information, job title, and equipment status. The homework tutoring camera can be flipped and mounted on the front of the base to take pictures of the homework on the desktop; The lens opening of the built-in optical engine base of the micro projector is located on the side of the base for projecting images to the outside. The sensors include an ambient light sensor and a sphere attitude sensor, which are used to automatically adjust the screen brightness and detect the sphere's rotation angle, respectively.

3. An interactive globe integrating intelligent tutoring and Internet of Things functions as described in claim 1, characterized in that, The dual-level access control system achieves mode switching through password, fingerprint, or facial recognition; Functionality is limited in the underage mode, while all functions are available in the adult mode. The function sets, AI service access permissions, screen mirroring permissions, and system settings permissions of the two modes are independent of each other, and switching modes requires higher-level authorization.

4. An interactive globe integrating intelligent tutoring and Internet of Things functions as described in claim 1, characterized in that, The multimodal intelligent interaction system integrates far-field voice recognition, touch screen operation, gesture recognition, and physical buttons, and sets up a unified interaction logic and priority processing mechanism.

5. The interactive globe as described in claim 1, characterized in that, The intelligent host module also integrates a screen projection module, a printing service module, and an IoT control module, supporting DLNA / Miracast screen projection, wireless printing, and control of smart home devices.

6. The interactive globe as claimed in claim 1, characterized in that, The AI ​​processing unit integrates an NPU and connects to a large-scale AI model in the cloud for local speech recognition, accelerated question image recognition, and cloud-based intelligent question answering and problem-solving guidance.

7. A method for operating an interactive globe integrating intelligent tutoring and Internet of Things (IoT) functions, characterized in that: An interactive globe integrating intelligent tutoring and Internet of Things (IoT) functions, as described in any one of claims 1-6, comprises the following steps: S1. Startup and Authorization Verification: The device is started by touch or voice wake-up. After startup, it enters the authorization verification process. The user completes the verification by password or biometric identification. The system enters the underage mode or adult mode according to the permission level. S2. In the corresponding mode, perform geographic learning, AI voice Q&A, homework tutoring, screen projection / printing or IoT control operations through multimodal interaction.

8. A method for operating an interactive globe integrating intelligent tutoring and Internet of Things (IoT) functions as described in claim 7, characterized in that, In the underage mode, only basic geography learning, homework tutoring, children's media playback, and filtered AI Q&A are available. It only supports authorized screen mirroring, the network is limited to the education server, and system-level settings are prohibited. In the adult mode, the full AI assistant, full-featured screen mirroring, full media resources, full IoT control, and all system settings are available.

9. The method for operating an interactive globe integrating intelligent tutoring and Internet of Things functions as described in claim 7, characterized in that, The homework tutoring process is as follows: turn on the homework tutoring function, point the camera at the question and take a picture. After OCR recognition and AI problem-solving model analysis, the solution steps are displayed on the screen, and the globe can be linked to locate and highlight relevant geographical areas.

10. The working method as described in claim 7, characterized in that, In adult mode, content can be projected to external display devices via Wi-Fi / Bluetooth, learning materials can be printed wirelessly, and smart home appliances can be controlled via voice commands through the IoT protocol module.

Images