Learning interaction method based on learning equipment, learning equipment and storage medium

Abstract

The embodiment of the invention discloses a learning interaction method based on learning equipment, the learning equipment and a storage medium, and the method comprises the steps: obtaining target information of a physical book, placing the physical book in an area scanned by the learning equipment, displaying a target page of the physical book on a screen of the learning equipment, the target information is obtained based on the target page; determining interaction information according to the target information, wherein the interaction information at least comprises a virtual animation; the virtual animation is displayed on the screen; according to the embodiment of the invention, the virtual animation can be displayed on the screen when the learning equipment displays the target page, so that the attraction to students is increased, the learning interest of the students is improved, and the learning effect is improved.

Description

Technical Field

[0001] This application relates to the technical field of learning devices, and in particular, to a learning interaction method, a learning device, and a storage medium based on a learning device. Background Art

[0002] Current learning devices incorporate artificial intelligence technology. They use the camera of the learning device to scan physical books, restore and display the book content on the learning device, and then enable functions such as the user operating the learning device by finger-pointing the physical book, which helps students' autonomous learning and improves learning efficiency. However, existing learning devices simply restore and display the scanned book content on the screen in a single display mode. During use, students are prone to feeling bored, it is difficult to maintain long-term learning interest, and it is not conducive to enhancing students' learning experience and learning effect. Summary of the Invention

[0003] Embodiments of this application disclose a learning interaction method, a learning device, and a storage medium based on a learning device, which can increase the attraction to students and improve students' learning interest through an interactive experience combining animation and sound effects, so as to improve the learning effect.

[0004] In a first aspect, embodiments of this application disclose a learning interaction method based on a learning device, and the method includes:

[0005] Obtain target information of a physical book, where the physical book is placed in an area scanned by the learning device, and the screen of the learning device displays a target page of the physical book, and the target information is obtained based on the target page;

[0006] Determine interaction information according to the target information, where the interaction information at least includes virtual animation;

[0007] Display the virtual animation on the screen.

[0008] In a second aspect, embodiments of this application disclose a learning device, which may include:

[0009] A first processing module, configured to obtain target information of a physical book, where the physical book is placed in an area scanned by the learning device, and the screen of the learning device displays a target page of the physical book, and the target information is obtained based on the target page;

[0010] A second processing module, configured to determine interaction information according to the target information, where the interaction information at least includes virtual animation;

[0011] A display module, configured to display the virtual animation on the screen.

[0012] In a third aspect, an embodiment of the present application discloses a learning device, which may include:

[0013] A memory storing executable program code;

[0014] A processor coupled to the memory;

[0015] The processor calls the executable program code stored in the memory and executes a learning interaction method based on a learning device disclosed in the first aspect of the embodiments of the present application.

[0016] In a fourth aspect, an embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute a learning interaction method based on a learning device disclosed in the first aspect of the embodiments of the present application.

[0017] In a fifth aspect, an embodiment of the present application discloses a computer program product, which, when running on a computer, causes the computer to execute some or all of the steps of any one of the methods in the first aspect.

[0018] In a sixth aspect, an embodiment of the present application discloses an application publishing platform for publishing a computer program product, wherein, when the computer program product runs on a computer, it causes the computer to execute some or all of the steps of any one of the methods in the first aspect.

[0019] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0020] In the embodiments of the present application, the learning device first obtains target information of a physical book placed in an area scanned by the learning device, and the screen of the learning device displays the target page of the physical book, and the target information is obtained based on the target page. Further, interaction information is determined according to the target information, and the interaction information at least includes virtual animations. Finally, the virtual animations are displayed on the screen of the learning device. By implementing the embodiments of the present application, when the learning device displays the target page, virtual animations can be displayed on the screen, increasing the attraction to students, improving students' learning interest, and thus improving the learning effect.

[0021] Further, by enhancing the interactive experience through virtual animations, a sense of immersion can be created, enhancing the fun and improving the learning experience of students.

[0022] Furthermore, since the virtual animations are determined according to the target information, and the target information is obtained based on the target page, not only can it visually stimulate the senses of students, but the relevant animations also help users better understand and remember knowledge points, improving the learning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0024] Figure 1 It is a diagram of the application scenario of the learning interaction method based on a learning device disclosed in the embodiments of the present application;

[0025] Figure 2 It is a schematic flowchart of the learning interaction method based on a learning device disclosed in Embodiment 1 of the present application;

[0026] Figure 3 It is a schematic diagram of the display position on the screen disclosed in the embodiments of the present application;

[0027] Figure 4 It is a schematic flowchart of the learning interaction method based on a learning device disclosed in Embodiment 2 of the present application;

[0028] Figure 5 It is a schematic flowchart of the learning interaction method based on a learning device disclosed in Embodiment 3 of the present application;

[0029] Figure 6 It is a schematic flowchart of the learning interaction method based on a learning device disclosed in Embodiment 4 of the present application;

[0030] Figure 7 It is a schematic diagram of the structure of the learning device disclosed in Embodiment 1 of the present application;

[0031] Figure 8 It is a schematic diagram of the structure of the learning device disclosed in Embodiment 3 of the present application. Detailed implementation manners

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", "third", "fourth", etc. in the description and claims of this application are used to distinguish different objects, rather than to describe a specific order. The terms "include" and "have" in the embodiments of this application and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0034] The learning device disclosed in the embodiments of this application may be a learning machine, a point reader, a tutoring machine, etc. Among them, the operating system of the learning device includes but is not limited to the Android operating system, the IOS operating system, the Symbian operating system, the Windows operating system, and so on. The execution subject of the embodiments of this application will be described in detail by taking the learning device as an example in combination with the accompanying drawings. It should be understood that this should not constitute any limitation to this application.

[0035] Please refer to Figure 1 , Figure 1 which is an application scenario diagram of the learning interaction method based on the learning device disclosed in the embodiments of this application. It can be understood that Figure 1 This is only an exemplary application scenario diagram of the learning interaction method disclosed in the embodiments of this application. Any other application scenarios obtained by optimizing or transforming on this Figure 1 basis and capable of implementing the technical solutions of this application fall within the protection scope of this application.

[0036] In Figure 1 the application scenario shown, the user can learn by using the learning device 10, and can place the physical book 20 in the area scanned by the learning device 10 ( Figure 1 schematically shown in front of the learning device). This area can be completely scanned and photographed by the shooting module 30 in the learning device 10. The learning device will restore and display the page of the scanned physical book on the screen, which can be on a specified area. Among them, restoring and displaying the page on the screen means completely restoring and displaying the content of the photographed page.

[0037] The photographing module 30 can be disposed on the side of the learning device 10 equipped with a screen. The photographing module 30 is used to photograph the physical book 20 to obtain the photographed page, and then restore and display it on the screen. A reflecting device (such as a reflector, a prism, or a convex lens, etc.) can also be disposed directly in front of the photographing module 30. The mirror surface of the reflecting device forms a preset angle with the lens surface of the photographing module 30. The reflecting device is used to change the optical path of the photographing module 30, so as to obtain the scanned and photographed page. By imaging the page obtained by the scanning and photographing of the photographing module 30 in the reflecting device, without manually changing the placement mode of the learning device 10, the photographing process can be simplified and the photographing efficiency can be improved.

[0038] Among them, after the learning device 10 opens a preset application (such as an interactive textbook application), after the physical book 20 is opened and placed in the scanned area, the current page of the physical book 20 is scanned and photographed by the photographing module 30, restored and displayed on the screen, and the virtual animation and / or page illustration element matching the unit theme element included in the current page is also obtained and displayed, and the sound effect audio can also be played simultaneously. Or, first open the physical book 20 and place it in the scanned area, then open the preset application on the learning device 10, scan the physical book 20 through the photographing module 30, display the current page on the screen, obtain the virtual animation and / or page illustration element matching the unit theme element included in the current page and display it, and the sound effect audio can also be played simultaneously. After that, if a page-turning operation or a click operation is performed on the physical book 20, the virtual animation will also be displayed on the screen, and the sound effect audio will be played. Among them, if it is a page-turning operation, the photographing module 30 will output and display the page obtained by scanning and photographing after page-turning on the screen, and obtain the unit theme element and / or page illustration element of the page after page-turning, then obtain the virtual animation and the sound effect audio, and display the virtual animation and play the sound effect audio on the screen. If it is a click operation, obtain the text information or page illustration element at the position corresponding to the click operation, obtain the matching virtual animation and sound effect audio according to the text information or page illustration element, and then display the virtual animation and play the sound effect audio on the screen, so as to combine the virtual animation and the sound effect on the learning device when matching and displaying the page of the physical book, increase the attraction to the user, increase the interest of learning, and improve the learning effect of students.

[0039] Among them, the physical book can be a textbook, a teaching material exercise book, a picture book, a literary novel, etc., and the electronic book corresponding to the physical book can also be synchronously stored in the learning device.

[0040] Optionally, the virtual animation and the sound effect audio can be preset according to the unit theme element, the page illustration element, the text information, etc. of the physical book, and stored locally or on the server side.

[0041] Combined with the above introduction, an embodiment of the present application discloses a learning interaction method, a learning device, and a storage medium based on a learning device, which are used to increase the attraction to students through animations when the learning device screen displays the physical book page, improve the learning interest of students, and improve the learning effect.

[0042] Next, the technical solution of the present application will be described in detail through specific embodiments in conjunction with the accompanying drawings.

[0043] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of the learning interaction method based on a learning device disclosed in Embodiment 1 of the present application; as Figure 2 shown, the learning interaction method based on the learning device may include:

[0044] 201. Obtain the target information of the physical book. The physical book is placed in the area scanned by the learning device, and the screen of the learning device displays the target page of the physical book. The target information is obtained based on the target page.

[0045] The execution subject of the embodiment of the present application is a learning device or a device, and the device can be built into the learning device. The physical book is placed in the area that the learning device can scan, such as Figure 1 placed directly in front of the learning device in the application scenario shown. The screen of the learning device displays the target page of the physical book, where the target page is the current page of the physical book and is displayed on the screen after being scanned by the shooting module of the learning device. Additionally, optionally, the embodiment of the present application can be implemented after the learning device opens a preset application.

[0046] Among them, the target information is obtained based on the target page, which can be the content on the target page or obtained according to the content of the target page. Optionally, the target information can be one or more of the theme elements, page illustration elements, and text information of the target page, and the theme element can further include one or more of the chapter theme element, unit theme element, and paragraph theme element. Exemplarily, the unit theme can be forest animals, and the corresponding page illustration elements can be animals such as birds, lions, and horses.

[0047] 202. Determine the interaction information according to the target information, and the interaction information at least includes a virtual animation.

[0048] Among them, the interaction information at least includes a virtual animation, and the virtual animation can be pre-set for the physical book and stored in the local memory or server of the learning device after establishing a corresponding relationship. Optionally, determining the interaction information according to the target information may include: searching for the interaction information matching the target information from the local memory or the server. By pre-setting the interaction information, the real-time calculation amount can be reduced, the processing efficiency can be improved, the performance can be improved, and the user experience can also be improved.

[0049] Optionally, the target information includes one or more of the theme elements, page illustration elements, and text information of the target page. Therefore, the virtual animation is an animation material that fits the above content of the target page, such as flowers, birds, fish, insects, the rising sun, insects chirping in the mountain stream, the rising sea tide, etc. For example, when the unit theme is forest animals and the corresponding page illustration elements are animals such as birds, lions, and horses, the virtual animation can be flowers, birds, fish, and insects to create a more surprising and immersive visual atmosphere.

[0050] 203. Display the virtual animation on the screen.

[0051] Among them, in step 203, the virtual animation is displayed on the screen. Optionally, the virtual animation can be superimposed and displayed on the target page displayed on the screen. Further, the background of the virtual animation can be transparent. When the virtual animation is superimposed and displayed on the target page, the overlapping part of the target page and the virtual animation is completely covered by the virtual animation, and the virtual animation is more vivid in visual effect, giving a feeling of coming at you, which helps users obtain a sense of immersion.

[0052] In some feasible implementation manners, a preset display position on the screen is used to display the page of the physical book. The display position can be geometric figures such as rectangles and squares corresponding to a preset size, and can be specifically set according to the shape of the physical book. When scanning the physical book, the pixels in the scanning area will be adjusted to the preset size, and then the physical book will be scanned to obtain the page corresponding to the preset size, and then directly output and displayed at the display position; or, when scanning the physical book, the pixels of the obtained target page are adjusted to the preset size of the display position and displayed at the display position. Through this implementation manner, a preset display position can be set, and the page of the physical book can be restored neatly and clearly on the screen, improving the visual experience.

[0053] Further, in some feasible implementation manners, the above display position can protrude upward by a preset height, such as Figure 3 As shown, the display position is preset in the middle of the screen. The display position is higher than other areas of the screen by a preset height, which can visually emphasize and highlight the displayed target page, improving the visual effect and reading experience.

[0054] It should be noted that the display of the virtual animation only lasts for a short duration (specified duration), which can be within 10 seconds, achieving the effect of being able to attract the user's attention without affecting the implementation of other functions.

[0055] Implementing the embodiments of the present application, the learning device first obtains the target information of the physical book. The physical book is placed in the area scanned by the learning device, and the screen of the learning device displays the target page of the physical book. The target information is obtained based on the target page. Further, interaction information is determined according to the target information, and the interaction information at least includes virtual animations. Finally, the virtual animations are displayed on the screen of the learning device. By implementing the embodiments of the present application, when the learning device displays the target page, virtual animations can be displayed on the screen, increasing the attraction to students, improving students' learning interest, and thus enhancing the learning effect.

[0056] Further, by enhancing the interaction experience through virtual animations, a strong sense of immersion can be created, the fun can be enhanced, and the learning experience of students can be improved.

[0057] Furthermore, since the virtual animations are determined according to the target information, and the target information is obtained based on the target page, it can not only stimulate students' senses visually, but also the relevant animations help users better understand and remember knowledge points, enhancing the learning effect.

[0058] Please refer to Figure 4 , Figure 4 which is a schematic flowchart of the learning interaction method based on the learning device disclosed in the second embodiment of the present application. As Figure 4 shown, the learning interaction method based on the learning device may include:

[0059] 401. When an interaction operation is detected and the screen displays the target page of the physical book, obtain the target information pointed to by the interaction operation from the target page. The physical book is placed in the area scanned by the learning device, and the screen of the learning device displays the target page of the physical book.

[0060] The execution subject of the embodiments of the present application is a learning device or apparatus, and the apparatus can be built into the learning device. The physical book is placed in the area that can be scanned by the learning device, such as Figure 1 placed directly in front of the learning device in the application scenario shown. The screen of the learning device displays the target page of the physical book, where the target page is the current page of the physical book and is displayed on the screen after being scanned by the shooting module of the learning device. Additionally, optionally, the embodiments of the present application can be implemented after the learning device opens a preset application.

[0061] Among them, the interaction operation may include a preset operation or a click operation. The preset operation further includes operations such as the user turning the page of the physical book, placing the physical book in the area scanned by the learning device and opening the preset of the learning device, and placing the physical book in the area scanned by the learning device for the first time after the preset application of the learning device is opened. The interaction operation points to the target page, and the target information of the target page is obtained based on the interaction operation.

[0062] Optionally, the target information includes one or more of the target page's theme elements, page illustration elements, and text information.

[0063] 402. Determine interaction information according to the target information, where the interaction information at least includes virtual animation.

[0064] Optionally, the interactive information also includes sound effects and audio corresponding to the virtual animation. The sound effects and audio are set to match the virtual animation and can enhance the display effect of the virtual animation from an auditory perspective. For example, the virtual animation is made of flowers, birds, fish, and insects, and the sound effects and audio can be the chirping sounds of birds, insects, etc.

[0065] 403. Display the virtual animation on the screen.

[0066] In step 403, the virtual animation is displayed on the screen. Optionally, the virtual animation can be superimposed on the target page displayed on the screen. Furthermore, the background of the virtual animation can be transparent. When the virtual animation is superimposed on the target page, the overlapping portion of the target page and the virtual animation is completely covered by the virtual animation. The virtual animation is visually more lifelike, giving a sense of being right in front of the user, which helps the user gain a sense of immersion.

[0067] Optionally, in combination with the description of step 402, when the virtual animation is displayed on the screen, the sound effect audio is also played, thereby increasing the appeal to students and improving students' learning interest through the interactive experience of the combination of virtual animation and sound effects, thereby improving learning outcomes.

[0068] Furthermore, the interactive experience of virtual animation and sound effects can create an overwhelming sense of immersion, enhance fun, and improve students' learning experience.

[0069] Furthermore, the interactive experience of virtual animation and sound effects can achieve multi-sensory stimulation, not only visual stimulation, but also auditory enjoyment. It can stimulate students' senses in multiple ways, help them better understand and remember knowledge points, and improve learning outcomes.

[0070] See also Figure 5 , Figure 5 This is a flow chart of the learning interaction method based on the learning device disclosed in Example 3 of this application; Figure 5 As shown, the learning interaction method based on the learning device may include:

[0071] 501. When a preset operation is detected, a target page of a physical book is displayed on a screen of a learning device, where the physical book is placed in an area scanned by the learning device.

[0072] The execution subject of the embodiments of this application is a learning device or apparatus, which can be built into the learning device. The physical book is placed in an area that can be scanned by the learning device, such as Figure 1 in the application scenario shown, placed directly in front of the learning device. The screen of the learning device displays the target page of the physical book, where the target page is the current page of the physical book and is displayed on the screen after being scanned by the shooting module of the learning device. Additionally, optionally, the embodiments of this application can be implemented after the learning device opens a preset application.

[0073] Among them, when the learning device detects a preset operation, it obtains the target page pointed to by the preset operation and displays the target page on the screen of the learning device.

[0074] In some feasible embodiments, step 501 may include:

[0075] Detect whether the user performs a page-turning operation on the physical book. After detecting the page-turning operation, obtain the page obtained after the physical book performs the page-turning operation as the target page; among them, it can be detected whether the user turns the page of the physical book. If it is a page turn, the page after the page turn needs to be used as the target page and then displayed on the screen.

[0076] It should be noted that in the above embodiments, it is executed after the physical book has been placed in the area scanned by the learning device and opened, and the learning device has opened the preset application.

[0077] Optionally, detecting whether the user performs a page-turning operation on the physical book may include: scanning a first series of frame images including the physical book through the shooting module of the learning device; analyzing the first series of frame images to obtain a first actual action, comparing the first actual action with a first preset action. If the first actual action is consistent with the first preset action, it is determined that the page-turning operation is detected. The first preset action is a demonstration action of the page-turning operation, where the page-turning operation may include a forward page-turning operation and a backward page-turning operation.

[0078] In some other feasible embodiments, step 501 may include:

[0079] After placing the physical book in the area scanned by the learning device, when it is detected that the learning device opens the preset application, take the current page of the physical book as the target page and display the target page on the screen; among them, after placing the physical book in the area scanned by the learning device and detecting that the learning device opens the preset application, scan the current page of the physical book through the shooting module of the learning device, and then use it as the target page to be displayed on the screen, and execute steps 502 - 504. That is, in the embodiments of this application, after the physical book is placed in the area and the preset application is opened, it will also trigger the execution of subsequent steps to achieve an animated display to improve the attractiveness.

[0080] In some other feasible embodiments, step 501 may include:

[0081] After the learning device opens a preset application, when it detects that the learning device scans a physical book from the area it scans, take the current page of the physical book as the target page and display the target page on the screen.

[0082] Among them, in the above embodiment, after the learning device opens the preset application, then open the physical book and place it in the area scanned by the learning device, the current page of the physical book will be scanned as the target page, the target page will be displayed on the screen, and subsequent steps will also be triggered to implement animated display to improve attractiveness.

[0083] In some other feasible embodiments, step 501 may include:

[0084] When the page of the physical book is displayed on the screen of the learning device, detect a sliding operation in the first preset direction on the screen, determine it as a page-turning operation, detect the corresponding e-book of the physical book, find the corresponding e-page (corresponding to the page displayed on the screen) on the e-book, and then after performing the page-turning operation corresponding to the sliding operation in the first preset direction, obtain the target e-page after page-turning as the target page and display the target page on the screen. In this embodiment, after the learning device displays the page of the physical book, the user can also perform page-turning on the screen of the learning device through a sliding operation. At this time, steps 502-504 will also be triggered to implement virtual animated display.

[0085] Among them, the first preset direction can be the left direction or the right direction. Among them, the left direction is used for backward page-turning of the e-book, and the right direction is used for forward page-turning of the e-book.

[0086] It should be noted that in the above embodiment, since the page-turning operation is performed on the screen, the target page displayed on the screen after performing the page-turning operation is not the current page of the physical book. If the physical book is page-turned next time, and the e-page corresponding to the page obtained after page-turning has been displayed on the screen, still scan the page of the physical book after page-turning as the target page, display it on the screen, and then perform subsequent operations to display the virtual animation.

[0087] It should be noted that the above steps 502-504 are executed after the page of the physical book is displayed on the screen of the learning device. When the learning device scans the cover of the physical book through the shooting module, it can just scan the cover and display it on the screen, or in addition to displaying the cover on the screen, also display the cover animation.

[0088] It should also be noted that the target page of the physical book is the page corresponding to the open state, which can be understood as corresponding to two pages, that is, the target page refers to the two pages currently in the open state. For example, pages 3 and 4 together are the target page. When the interaction operation is a page-turning operation, if the pages of the physical book before page-turning are pages 3 and 4, then when performing a forward page-turning operation, the pages obtained after page-turning are pages 1 and 2, and then pages 1 and 2 are used as the target pages and displayed on the screen; when performing a backward page-turning operation, the pages obtained after page-turning are pages 5 and 6, and then pages 5 and 6 are used as the target pages and displayed on the screen.

[0089] 502. Obtain first element information from the target page as target information.

[0090] Optionally, the first element information includes at least unit theme elements and / or page illustration elements. Obtaining first element information from the target page as target information includes: identifying the target page number of the target page, and in the corresponding relationship between the page number and the element information, obtaining the first element information corresponding to the target page number as the target information. In this embodiment, the corresponding relationship between the page number and the element information can be established and saved in advance, which can improve the real-time processing efficiency.

[0091] Among them, the unit theme element in the first element information can be to extract the unit theme or unit text content in the corresponding page number, then analyze it to obtain the unit theme element, and then extract the illustration in the target page number as the page number illustration element.

[0092] Optionally, since the target page number is a double page, obtaining first element information from the target page as target information includes: by default, obtaining first element information from the left page or the right page of the target page number as the target information. In this embodiment, directly specifying which page to obtain the first element information from is simpler.

[0093] Another option is that obtaining first element information from the target page as target information includes: if the preset operation is a page-turning operation, if it is a backward page-turning operation, obtain the left page of the target page number obtained after page-turning, and then obtain the first element information from the left page as the target information; if it is a forward page-turning operation, obtain the right page of the target page number obtained after page-turning, and then obtain the first element information from the right page as the target information. In this embodiment, the first element information can be obtained according to the order of the pages read first after page-turning, which is more in line with the content read first.

[0094] Another option is that obtaining first element information from the target page as target information includes: if the preset operation is an operation other than the page-turning operation, the entire target page number can be comprehensively considered, and all unit theme elements and / or page illustration elements of the target page number are obtained as target information.

[0095] 503. Determine the interaction information according to the target information, where the interaction information includes at least virtual animations.

[0096] Optionally, the interaction information further includes the sound effect audio corresponding to the virtual animation. The sound effect audio is set to match the virtual animation and can enhance the display effect of the virtual animation aurally. Exemplarily, if the virtual animation is a material of flowers, birds, fish, and insects, the sound effect audio can be the chirping sounds of birds and insects.

[0097] 504. Display the virtual animation on the screen.

[0098] Optionally, in combination with the description in step 503, when the virtual animation is displayed on the screen, the sound effect audio is also played, so as to increase the attraction to students and improve their learning interest through the interactive experience of combining the virtual animation and the sound effect, thereby improving the learning effect.

[0099] Therefore, by implementing the embodiments of the present application, when the user performs a page-turning operation on the physical book or the page of the physical book is first scanned and displayed, the interactive experience can be improved by combining the virtual animation and the sound effect audio, increasing the attraction to students and improving their learning interest, thereby improving the learning effect. Further, the interactive experience of the virtual animation and the sound effect can create an immersive feeling that comes at you, enhance the fun, and improve the learning experience of students. Further, the interactive experience of the virtual animation and the sound effect can also achieve multi-sensory stimulation, not only visual stimulation but also auditory enjoyment, which can stimulate the senses of students in multiple directions, better understand and remember knowledge points, and improve the learning effect.

[0100] Please refer to Figure 6 , Figure 6 , which is a schematic flowchart of the learning interaction method based on a learning device disclosed in Embodiment 3 of the present application; as Figure 6 shown, the learning interaction method based on the learning device may include:

[0101] 601. When the target page is displayed on the screen of the learning device, detect whether the user performs a click operation on the current page of the physical book, where the current page is the target page.

[0102] The execution subject of the embodiments of the present application is a learning device or apparatus, and the apparatus can be built into the learning device. The physical book is placed in an area that can be scanned by the learning device, such as Figure 1 shown in the application scenario, placed directly in front of the learning device. The screen of the learning device displays the target page of the physical book, where the target page is the current page of the physical book and is displayed on the screen after being scanned by the shooting module of the learning device. Additionally, optionally, the embodiments of the present application can be implemented after the learning device opens a preset application.

[0103] Optionally, detecting whether the user performs a click operation on the current page of the physical book includes: scanning a second series of frame images containing the physical book through the shooting module of the learning device; analyzing the second series of frame images to obtain a second actual action, comparing the second actual action with a second preset action, and if the second actual action is consistent with the second preset action, determining that a click operation is detected, where the second preset action is a demonstration action of the click operation. Through this implementation, the shooting module of the learning device can accurately detect the click operation.

[0104] In some feasible implementation manners, step 601 may further include: when the target page is displayed on the screen of the learning device, detecting whether a click operation occurs on the screen of the learning device. In this implementation manner, the user can also perform a click operation based on the target page displayed on the screen of the learning device, and then trigger the execution of steps 602-604 to implement the corresponding functions.

[0105] 602. Obtain second element information corresponding to the position of the click operation from the target page as target information.

[0106] Wherein, the second element information at least includes text and / or page illustration elements.

[0107] Optionally, in step 602, through image recognition technology, the position corresponding to the click operation and the text and / or page illustration elements within the area corresponding to the position can be recognized as target elements.

[0108] Furthermore, the pages of the physical book can be pre-divided according to each paragraph of text or each sentence, etc. The specific division is based on the presented paragraph theme or sentence meaning, and the illustration elements related to the page and the paragraph theme or sentence meaning are obtained. The paragraph theme or sentence meaning is associated with the related illustration theme and stored in the learning device or server. Optionally, obtaining the second element information corresponding to the position of the click operation from the target page as target information in step 602 includes: recognizing the text information at the position of the click operation, obtaining whether there are page illustration elements related to the literary information, and if so, obtaining the related page illustration elements and the text information as the second element information. Through this implementation, the second element information can be quickly obtained, improving the processing efficiency.

[0109] 603. Determine interaction information according to the target information, where the interaction information at least includes virtual animations.

[0110] Optionally, the interaction information further includes the sound effect audio corresponding to the virtual animation, and the sound effect audio is set to match the virtual animation, which can enhance the display effect of the virtual animation auditorily. Exemplarily, if the virtual animation is of flower, bird, fish, and insect materials, the sound effect audio can be the chirping sounds of birds and insects.

[0111] 604. Display virtual animations on the screen.

[0112] Optionally, in combination with the description in step 603, when displaying virtual animations on the screen, the sound effect audio is also played, so as to increase the attraction to students and improve their learning interest through the interactive experience of combining virtual animations and sound effects, thereby improving the learning effect.

[0113] In some alternative embodiments, after performing steps 601 - 604, the function corresponding to the click operation is executed in the application currently opened on the learning device. For example, obtain the interpretation materials associated with the target information and output them to the user in the output manner corresponding to the interpretation materials, where the interpretation materials are at least one of literal interpretation analysis, the deep meaning of the paragraph where the text is located, follow - up audio, word - formation and sentence - making, and picture - description. Exemplarily, when the user activates the point - reading function of the learning device and clicks on the page of the physical book with a point - reading pen or finger, before the learning device executes the point - reading function, it first displays virtual animations and plays sound effect audio to attract the user's attention, and then plays the preset voice corresponding to the target information after the virtual animations are displayed and the sound effect audio is played.

[0114] In some feasible embodiments, a preset virtual pet is output and displayed at the virtual pet position on the screen; when a touch operation on the preset virtual pet is detected, the effect animation and effect audio of the preset virtual pet are obtained; within a preset duration, the effect animation is used to replace the virtual pet output and displayed at the virtual pet position, and the effect audio is played; after the preset duration, the preset virtual pet is restored and displayed at the virtual pet position.

[0115] Among them, at least the display position of the page and the virtual pet position are divided on the screen. A preset virtual pet is displayed at the virtual pet position, and the user can interact with the preset virtual pet, that is, the user can touch the preset virtual pet, and then obtain the effect animation and effect audio of the virtual pet, which replace the preset virtual pet and are displayed at the virtual pet position. The output duration of the effect animation and effect audio is the preset duration. After the preset duration, the effect animation disappears, and then the virtual pet is restored and displayed at the virtual pet position. It can be understood that the effect animation and effect audio of the preset virtual pet can be pre - associated and saved.

[0116] Furthermore, the virtual pet position can fixedly output and display a certain preset virtual pet, or obtain a more matching preset virtual pet according to the subject classification of the physical book. For example, when the physical book starts to be displayed on the screen, obtain the subject classification of the physical book, and select a matching preset virtual pet from several preset virtual pets pre - saved according to the subject classification, and output and display it at the virtual pet position. The subject classification can be Chinese, Mathematics, Social Studies, Science, etc.

[0117] Of course, function buttons corresponding to some applications are also displayed on the task bar or other positions on the screen, such as the point-reading mode, interactive textbook, text following, etc. The embodiments of the present application can be integrated into these applications. For example, the embodiments of the present application can be implemented under the function of enabling the interactive textbook.

[0118] Optionally, some cards can also be displayed on the screen, such as word cards, illustration cards, etc. Users can also implement corresponding functions by interacting with the cards.

[0119] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of the learning device disclosed in Embodiment 1 of the present application; as Figure 7 shown, the learning device may include:

[0120] A first processing module 701, configured to obtain target information of a physical book, where the physical book is placed in an area scanned by the learning device, and a target page of the physical book is displayed on the screen of the learning device, and the target information is obtained based on the target page;

[0121] A second processing module 702, configured to determine interaction information according to the target information, where the interaction information at least includes virtual animations;

[0122] A display module 703, configured to display the virtual animations on the screen.

[0123] In some feasible embodiments, the above interaction information further includes sound effect audio corresponding to the virtual animations, and the display module 703 is further configured to play the sound effect audio when displaying the virtual animations on the screen.

[0124] In some feasible embodiments, the specific manner for the first processing module 701 to obtain the target information of the physical book is: when an interaction operation is detected and the target page is displayed on the screen, obtain the target information pointed to by the interaction operation from the target page.

[0125] Further, the interaction operation is a preset operation, and the display module 703 is further configured to, when the first processing module 701 detects the interaction operation and before obtaining the target information pointed to by the interaction operation from the target page when the target page is displayed on the screen, display the target page on the screen when the preset operation is detected;

[0126] Furthermore, when the first processing module 701 detects an interaction operation and the target page is displayed on the screen, the specific way to obtain the target information pointed to by the interaction operation from the target page is as follows: when the preset operation is detected and the target page is displayed on the screen, first element information is obtained from the target page as the target information.

[0127] Further optionally, when the display module 703 detects the preset operation, the specific way to display the target page on the screen is as follows:

[0128] Detect whether the user performs a page-turning operation on the physical book. After the page-turning operation is detected, the page obtained after the physical book performs the page-turning operation is used as the target page.

[0129] Or, when the display module 703 detects the preset operation, the specific way to display the target page on the screen is as follows:

[0130] After placing the physical book in the area scanned by the learning device, when it is detected that the learning device opens the preset application, the current page of the physical book is used as the target page, and the target page is displayed on the screen.

[0131] Or, when the display module 703 detects the preset operation, the specific way to display the target page on the screen is as follows:

[0132] After the learning device opens the preset application, when it is detected that the learning device scans the physical book from the area it scanned, the current page of the physical book is used as the target page, and the target page is displayed on the screen.

[0133] In some other embodiments, the interaction operation is a click operation. Before the first processing module 701 obtains the target information pointed to by the interaction operation from the target page when detecting the interaction operation and the target page is displayed on the screen, it also detects whether the user performs a click operation on the current page of the physical book, and the current page is the target page.

[0134] Furthermore, when the first processing module 701 detects the interaction operation and the target page is displayed on the screen, the specific way to obtain the target information pointed to by the interaction operation from the target page is as follows:

[0135] When the click operation is detected and the target page is displayed on the screen, second element information corresponding to the position of the click operation is obtained from the target page as the target information.

[0136] Further, when the interaction operation is the click operation, and after the virtual animation is displayed on the screen, the first processing module 701 is further configured to execute the function corresponding to the click operation in the application currently opened by the learning device.

[0137] In some alternative embodiments, the display module 703 is further configured to display a preset virtual pet at the virtual pet position on the screen; furthermore, the second processing module 702 is further configured to obtain the effect animation and effect audio of the preset virtual pet when a touch operation on the preset virtual pet is detected; finally, the display module 703 is further configured to, within a preset duration, replace the preset virtual pet with the effect animation for output display at the virtual pet position, and play the effect audio; after the preset duration, the preset virtual pet is restored for display at the virtual pet position.

[0138] Please refer to Figure 8 , Figure 8 which is a schematic structural diagram of the learning device disclosed in Embodiment 3 of the present application; as Figure 8 shown, the learning device may include:

[0139] A memory 801 storing executable program code;

[0140] A processor 802 coupled to the memory 801;

[0141] Wherein, the processor 802 calls the executable program code stored in the memory 801 and executes Figure 2 , Figures 4 to 6 Some steps of any one of the learning interaction methods based on the learning device.

[0142] The embodiment of the present application also discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute Figure 2 , Figures 4 to 6 Any one of the learning interaction methods based on the learning device disclosed.

[0143] The embodiment of the present application also discloses a computer program product, when the computer program product runs on a computer, enabling the computer to execute Figure 2 , Figures 4 to 6 Some or all of the steps of any one of the methods disclosed.

[0144] The embodiment of the present application also discloses an application publishing platform for publishing a computer program product, wherein when the computer program product runs on a computer, enabling the computer to execute Figure 2 , Figures 4 to 6 Some or all of the steps of any one of the methods disclosed.

[0145] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disc memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0146] The above has introduced in detail a learning interaction method, a learning device, and a storage medium based on a learning device disclosed in the embodiments of the present invention. Specific examples are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A learning interaction method based on a learning device, characterized in that, The method includes: Obtaining target information of a physical book, where the physical book is placed in an area scanned by the learning device, the screen of the learning device displays a target page of the physical book, and the target information is obtained based on the target page; Determining interaction information according to the target information, where the interaction information includes at least virtual animations; Displaying the virtual animations on the screen.

2. The method according to claim 1, characterized in that The obtaining of the target information of the physical book includes: When an interaction operation is detected and the screen displays the target page, obtaining the target information pointed to by the interaction operation from the target page.

3. The method according to claim 2, characterized in that The interaction operation is a preset operation. Before obtaining the target information pointed to by the interaction operation from the target page when the interaction operation is detected and the screen displays the target page, the method further includes: When the preset operation is detected, displaying the target page on the screen; The obtaining of the target information pointed to by the interaction operation from the target page when the interaction operation is detected and the screen displays the target page includes: When the preset operation is detected and the screen displays the target page, obtaining first element information from the target page as the target information.

4. The method according to claim 3, wherein The displaying of the target page on the screen when the preset operation is detected includes: Detecting whether the user performs a page-turning operation on the physical book. After the page-turning operation is detected, obtaining the page obtained after the physical book performs the page-turning operation as the target page; or After the physical book is placed in the area scanned by the learning device, when it is detected that the learning device opens a preset application, taking the current page of the physical book as the target page and displaying the target page on the screen; or After the learning device opens the preset application, when it is detected that the learning device scans the physical book from the area it scans, taking the current page of the physical book as the target page and displaying the target page on the screen.

5. The method according to claim 2, characterized in that The interaction operation is a click operation. Before obtaining the target information pointed to by the interaction operation from the target page when the interaction operation is detected and the screen displays the target page, the method further includes: Detecting whether the user performs a click operation on the current page of the physical book, where the current page is the target page; The obtaining of the target information pointed to by the interaction operation from the target page when the interaction operation is detected and the screen displays the target page includes: When the click operation is detected and the screen displays the target page, obtaining second element information corresponding to the position of the click operation from the target page as the target information.

6. The method according to claim 1, wherein The interaction information further includes: sound effect audio corresponding to the virtual animations; the method further includes: Playing the sound effect audio while displaying the virtual animations on the screen.

7. The method according to claim 5, characterized in that The method further includes: When the interaction operation is the click operation, and after the virtual animations are displayed on the screen, performing the function corresponding to the click operation in the application currently opened by the learning device.

8. The method according to claim 1, characterized in that, The method further includes: displaying a preset virtual pet at the virtual pet position on the screen; when a touch operation on the preset virtual pet is detected, obtaining an effect animation and an effect audio of the preset virtual pet; within a preset duration, replacing the preset virtual pet with the effect animation for output display at the virtual pet position and playing the effect audio; after the preset duration, restoring the display of the preset virtual pet at the virtual pet position.

9. A learning device, characterized in that, including: a first processing module, configured to obtain target information of a physical book, where the physical book is placed in an area scanned by the learning device, and a target page of the physical book is displayed on the screen of the learning device, and the target information is obtained based on the target page; a second processing module, configured to determine interaction information according to the target information, where the interaction information at least includes a virtual animation; a display module, configured to display the virtual animation on the screen.

10. A learning device, characterized in that, including: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the learning interaction method based on a learning device according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps of the method according to any one of claims 1-8 are implemented.

Images