A stereoscopic knowledge point explanation method and system

Abstract

The application belongs to the field of auxiliary learning, and particularly relates to a three-dimensional knowledge point explanation method and system. A target to be recognized is placed in a desktop projection area, a target image in the desktop projection area is collected, a multi-modal recognition is performed on the target to be recognized in the desktop projection area based on a preset classification library, specific physical objects and core theme content of the physical objects recognized are output, and an identification result is obtained by sorting. A basic popular science introduction of the corresponding physical object is output based on the identification result, and a basic reading knowledge point is generated. Based on the core theme content, corresponding data in a third-party knowledge base is called, and core data knowledge points in different dimensions are generated. Panoramic encyclopedia content is synchronously loaded and linked to the desktop projection to generate a VR three-dimensional visual explanation picture, and the picture is output to the desktop projection area. The method realizes intelligent recognition and knowledge expansion of multiple objects such as textbooks, books, picture books, physical objects and handwritten topics, and is a core supporting capability of a product "paper-electricity fusion and active exploration learning".

Description

Technical Field

[0001] This application belongs to the field of learning assistance, and specifically relates to a three-dimensional method and system for explaining knowledge points. Background Technology

[0002] While existing interactive reading products possess text recognition and text-to-speech functions, their knowledge point explanation methods have significant shortcomings. These are as follows:

[0003] 1. Existing equipment only outputs single-layer shallow information, lacks a step-by-step, layered explanation logic, cannot match students' gradual learning pace, and makes it difficult to build a complete knowledge system.

[0004] 2. Existing interactive reading products rely solely on text and voice for knowledge output, lacking structured formats and resulting in fragmented knowledge that is difficult to organize and memorize. Furthermore, the devices employ a uniform explanation logic, failing to adapt explanation strategies to different materials such as textbooks, picture books, handwritten content, and physical objects, leading to poor scenario adaptability.

[0005] 3. Most existing products do not support physical object recognition for popular science education, relying solely on textual explanations of physical object knowledge, which are abstract and boring; moreover, recognition only supports single-line scanning, making it difficult to quickly select and crop materials, resulting in low efficiency in locating knowledge points.

[0006] In summary, existing methods of explaining knowledge points are limited in hierarchy, use a single medium, lack contextual adaptation and two-way interaction, and cannot achieve systematic, tiered teaching. There is an urgent need to provide a progressive method for explaining knowledge points that is adaptable to multiple types of recognition objects. Summary of the Invention

[0007] To address the aforementioned issues, this application provides a three-dimensional knowledge point explanation method and system to resolve the problems of existing knowledge point explanation methods having a thin hierarchy, a single carrier, and lacking scenario adaptation and two-way interaction.

[0008] The technical solution of this application is: a three-dimensional method for explaining knowledge points, including:

[0009] Collect a certain amount of physical data, including: three-dimensional data of daily necessities, learning items and animals and plants, establish a preset classification library, and label each physical object with its standard name, family or category label and primary visual characteristics;

[0010] The target to be identified is placed on the desktop projection area, and the target image in the desktop projection area is acquired. Based on the preset classification library, the target to be identified in the desktop projection area is recognized in a multimodal manner, and the specific object and the core theme content of the object are output. The recognition results are then sorted out.

[0011] Based on the recognition results, basic popular science introductions of the corresponding real objects are output, and basic recognition knowledge points are generated.

[0012] Based on the core theme, corresponding data from third-party knowledge bases are called to generate core data knowledge points in different dimensions, including background knowledge supplements, core knowledge point memory cards and graphic thematic explanation courseware.

[0013] Based on basic literacy knowledge points and core data, panoramic encyclopedia content is loaded synchronously and linked with desktop projection to generate VR stereoscopic visualization explanation screens, which are then output to the desktop projection area; learners can learn knowledge points by manually clicking or through voice interaction.

[0014] Preferably, multimodal recognition is performed on objects within the desktop projection area, specifically:

[0015] After placing the target to be identified on the desktop projection area, the desktop background and the target object foreground are separated by the image semantic segmentation algorithm. Desktop texture, books, and shadow areas are removed, and object data is extracted.

[0016] Visual feature data are extracted from physical data, including shape and size, surface texture, primary and secondary color schemes, and local key features, to obtain multidimensional visual features;

[0017] The multidimensional visual features are matched with the primary visual features in the preset classification library to obtain the corresponding physical objects. The standard name, family, genus, or category label of the physical objects are then output, and the recognition is completed.

[0018] After recognition is completed, the system synchronously calls up the physical encyclopedia database to load full-dimensional popular science content, and outputs the explanation content in conjunction with the display screen.

[0019] Preferably, the multidimensional visual features are matched with the primary visual features in a preset classification library, specifically as follows:

[0020] If a single visual feature has a matching confidence level below the threshold, the projection depth auxiliary information is retrieved for secondary verification. The three-dimensional contour features of the real object are supplemented and re-compared to output the final accurately identified object.

[0021] Preferably, the physical objects are divided into primary objects with no readable content on their surfaces and secondary objects with readable content on their surfaces. The secondary objects include textbooks, books, and picture books. The surfaces of the secondary objects are further scanned by a scanning device, and text and image recognition is performed to obtain secondary visual features, which are then used as the core theme content.

[0022] Preferably, the secondary physical surface is further scanned using a scanning device, specifically as follows:

[0023] After the scanning device is started, the effective interactive recognition area is delineated by desktop projection. After placing an open physical object with readable content, the posture of the book page is corrected by adjusting the projection frame of the scanning device, and the open content of the physical object is collected.

[0024] The semantic segmentation model is used to process the contents of the physical object, including: dividing the area into the vocabulary list, the main text of the lesson, the after-class exercises, the short writing exercises, and the illustration area, and removing the illustrations and decorative patterns; at the same time, the line spacing and paragraph segmentation information of the original layout are restored, and the hierarchical structure of the pinyin column, the Chinese character column, and the question and answer column are distinguished to obtain the text recognition content.

[0025] The system calls upon third-party knowledge bases, including: vocabulary databases and text corpora from primary and secondary school textbooks, and performs corresponding verification between the text recognition content and the third-party knowledge bases to match the original texts of standard textbooks or extracurricular reading materials for the corresponding grade level; at the same time, it provides labels for the text recognition content.

[0026] Preferably, when using a semantic segmentation model to process the unfolded content of the secondary physical object:

[0027] The text lines are skewed and segmented to break the entire page of text into single-line text sequences, thus avoiding cross-line recognition errors.

[0028] For bulleted sentences such as after-class Q&A and short writing exercises, retain the original bullet points and line breaks, and restore the format of the questions and guiding example sentences;

[0029] When setting a single-match confidence level, if the confidence level of the text recognition content is insufficient when verifying its correspondence with a third-party knowledge base, it is determined to be non-text. The projection depth information is combined to distinguish between book pages and the lower desktop, eliminating interference from paper wrinkles and shadows.

[0030] Preferably, when collecting the contents of the second-level physical object, two collection triggering methods are supported: one is to use the matching stylus to select the target text paragraph to trigger collection, and the other is to directly scan the whole page and then crop the local image and text area to collect high-definition RGB page images.

[0031] For the new characters with pinyin, a pinyin-Chinese character linkage recognition branch is adopted: first, the pinyin characters above are recognized, and then the corresponding Chinese characters below are matched to correct the recognition errors of similar-looking characters and pinyin tones; for the pure Chinese text sections of the textbook, an OCR model adapted to the font of primary and secondary school textbooks is used to recognize each character and output the complete text with punctuation and line break format.

[0032] Preferably, the panoramic encyclopedia content is loaded simultaneously and linked to a desktop projection to generate a VR stereoscopic visual explanation screen, specifically as follows:

[0033] The system retrieves the 3D data of the corresponding real object from the preset classification library to reconstruct the 3D model, and matches the model texture and structural details with panoramic encyclopedia content. The reconstructed 3D model of the real object is then mapped onto the desktop projection interactive area to generate a VR stereoscopic visualization explanation screen. At the same time, basic reading knowledge points, background knowledge, memory cards, and special topic explanation courseware are superimposed and projected around the VR stereoscopic visualization explanation screen.

[0034] Another technical solution of this application is: a three-dimensional knowledge point explanation system, including:

[0035] The classification library creation module is used to collect a certain amount of physical data, including: daily necessities, learning items and three-dimensional data of plants and animals, to create a preset classification library, and to mark the standard name, family or category label and primary visual features of each physical object;

[0036] The image acquisition module is used to place the target to be identified on the desktop projection area, acquire the target image in the desktop projection area, perform multimodal recognition on the target to be identified in the desktop projection area based on a preset classification library, output the specific object identified and the core theme content of the object, and sort out the recognition results.

[0037] The knowledge popularization module is used to output basic popular science introductions of corresponding objects based on the recognition results, and to generate basic recognition knowledge points.

[0038] The extended output module is used to generate core data knowledge points in different dimensions by calling corresponding data from third-party knowledge bases based on core theme content, including background knowledge supplements, core knowledge point memory cards, and graphic and text-based thematic explanation courseware.

[0039] The knowledge explanation module is used to load panoramic encyclopedia content and generate VR stereoscopic visualization explanation screens based on basic recognition knowledge points and core data, and output them to the desktop projection area. Learners can learn knowledge points by manually clicking or through voice interaction.

[0040] Preferably, in the image acquisition module, after the target to be identified is placed on the desktop projection area, the desktop background and the target object foreground are separated by an image semantic segmentation algorithm, the desktop texture, books, and shadow areas are removed, and the object data is extracted.

[0041] Visual feature data are extracted from physical data, including shape and size, surface texture, primary and secondary color schemes, and local key features, to obtain multidimensional visual features;

[0042] The multidimensional visual features are matched with the primary visual features in the preset classification library to obtain the corresponding physical objects. The standard name, family, genus, or category label of the physical objects are then output, and the recognition is completed.

[0043] After recognition is completed, the system synchronously calls up the physical encyclopedia database to load full-dimensional popular science content, and outputs the explanation content in conjunction with the display screen.

[0044] Preferably, in the image acquisition module, multidimensional visual features are matched with first-level visual features in a preset classification library, specifically as follows:

[0045] If a single visual feature has a matching confidence level below the threshold, the projection depth auxiliary information is retrieved for secondary verification. The three-dimensional contour features of the real object are supplemented and re-compared to output the final accurately identified object.

[0046] The three-dimensional knowledge point explanation method and system proposed in this application have the following advantages:

[0047] By combining dual-mode image acquisition with a multimodal recognition mechanism, it can uniformly complete the recognition of all categories of objects, including textbook text and illustrations, picture books, handwritten text, fruits / stationery / plants, etc. It can accurately correct the light and shadow of the pages and tilt distortion to stabilize the recognition of printed textbook content. It can also achieve accurate classification of three-dimensional objects through foreground segmentation and multi-dimensional visual feature fusion. It realizes intelligent recognition and knowledge expansion of multiple objects such as textbooks, books, picture books, physical objects, and handwritten topics. It is the core supporting capability of the product's "paper and electronic integration and active exploratory learning".

[0048] It constructs a hierarchical and progressive knowledge point explanation chain consisting of basic recognition, background expansion, knowledge cards, thematic courseware, and voice interactive Q&A. It no longer simply outputs superficial explanations, but breaks down knowledge according to the learning logic from simple to complex. At the same time, it matches exclusive explanation strategies for textbooks, picture books, handwritten content, and real objects, and differentiates the output content by distinguishing in-class test points, story creation, thematic summarization, and three-dimensional science popularization. It is suitable for various learning scenarios such as previewing, reading, and physical exploration, and meets the hierarchical learning needs of different stages. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the overall process of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are only some, not all, of the embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0051] The first aspect of this application provides a three-dimensional method for explaining knowledge points, such as... Figure 1 As shown, it includes:

[0052] Step S100: Collect a certain amount of physical data, including: three-dimensional data of daily necessities, learning items and plants and animals, establish a preset classification library, and mark the standard name, family or category label and primary visual characteristics of each physical object.

[0053] Step S200: Place the target to be identified on the desktop projection area, collect the target image in the desktop projection area, perform multimodal recognition on the target to be identified in the desktop projection area based on the preset classification library, output the specific object identified and the core theme content of the object, and sort out the recognition results.

[0054] Preferably, multimodal recognition is performed on objects within the desktop projection area, specifically:

[0055] After placing the target to be identified on the desktop projection area, the desktop background and the target object foreground are separated by the image semantic segmentation algorithm. Desktop texture, books, and shadow areas are removed, and object data is extracted.

[0056] Visual feature data are extracted from physical data, including shape and size, surface texture, primary and secondary color schemes, and local key features, to obtain multidimensional visual features;

[0057] The multidimensional visual features are matched with the primary visual features in the preset classification library to obtain the corresponding physical objects, such as apples, books, teacups, etc. The standard name, family or category label of the physical object is output, and the recognition is completed.

[0058] After recognition is completed, the system synchronously calls up the physical encyclopedia database to load full-dimensional popular science content, and outputs the explanation content in conjunction with the display screen.

[0059] By semantic segmentation, distracting backgrounds such as desktops, shadows, and other books are removed, and the actual object data is extracted separately, thus purifying the image to be recognized and significantly reducing the impact of a cluttered desktop environment on recognition accuracy.

[0060] In a specific example, when a pencil, eraser, and book are placed on a table at the same time, the system can automatically remove the background of the book and extract the outline texture of the pencil separately, without mistakenly identifying the book pattern as stationery.

[0061] Preferably, the multidimensional visual features are matched with the primary visual features in a preset classification library, specifically as follows:

[0062] If a single visual feature has a matching confidence level below the threshold, the projection depth auxiliary information is retrieved for secondary verification. The three-dimensional contour features of the real object are supplemented and re-compared to output the final accurately identified object.

[0063] When the confidence threshold for feature matching is set, if the accuracy of matching a single two-dimensional visual feature is insufficient, projection depth information is introduced to supplement the secondary verification of the three-dimensional contour, thus making up for the shortcoming of the loss of three-dimensional structural information in the planar image.

[0064] Preferably, the physical objects are divided into primary objects with no readable content on their surfaces and secondary objects with readable content on their surfaces. The secondary objects include textbooks, books, and picture books. The surfaces of the secondary objects are further scanned by a scanning device, and text and image recognition is performed to obtain secondary visual features, which are then used as the core theme content.

[0065] In a specific example, placing a picture book not only identifies the physical picture book but also allows the extraction of the illustrations and text as the core theme.

[0066] A text and image recognition process has been added for textbooks and picture books, extracting secondary visual features as the core theme for explanation, achieving dual recognition compatibility of three-dimensional objects and two-dimensional text.

[0067] Preferably, for physical objects with readable content on their surface, text and image recognition are performed, specifically using the following method:

[0068] After the scanning device is started, the effective interactive recognition area is delineated by desktop projection. After placing an open secondary physical object with readable content, the posture of the book page is corrected by adjusting the projection frame of the scanning device, and the open content of the secondary physical object is collected.

[0069] The semantic segmentation model is used to process the contents of the second-level physical object, including: dividing the vocabulary list, text, after-class exercises, short writing exercises, and illustration areas, and removing illustrations and decorative patterns; at the same time, restoring the line spacing and paragraph segmentation information of the original layout, distinguishing the hierarchical structure of the pinyin column, Chinese character column, and question and answer column, and obtaining the text recognition content.

[0070] The system calls upon third-party knowledge bases, including: vocabulary databases and text corpora from primary and secondary school textbooks, and performs corresponding verification between the text recognition content and the third-party knowledge bases to match the original texts of standard textbooks or extracurricular reading materials for the corresponding grade level; at the same time, it provides labels for the text recognition content.

[0071] Preferably, when using a semantic segmentation model to process the unfolded content of the secondary physical object:

[0072] The text lines are skewed and segmented to break the entire page of text into single-line text sequences, thus avoiding cross-line recognition errors.

[0073] For bulleted sentences such as after-class Q&A and short writing exercises, retain the original bullet points and line breaks, and restore the format of the questions and guiding example sentences;

[0074] When setting a single-match confidence level, if the confidence level of the text recognition content is insufficient when verifying its correspondence with a third-party knowledge base, it is determined to be non-text. The projection depth information is combined to distinguish between book pages and the lower desktop, eliminating interference from paper wrinkles and shadows.

[0075] Text line slant correction and line segmentation are implemented to avoid cross-line text recognition errors; exercise bullet points and paragraph formatting are fully preserved, and the output content conforms to the original reading logic of the textbook.

[0076] For tilted fonts caused by page turning, further identification is performed. Specifically, the page area is extracted, the overall tilt angle of the page is detected, and the entire page is straightened through rotation and perspective transformation. Then, the text on the page is split into a single-line sequence, and minor skews are detected and corrected line by line. At the same time, projection data is retrieved to identify the folded and warped positions of the page, and grid deformation is used to repair the distortion of curved text. After the distortion of curved text is corrected, the pinyin and Chinese characters are aligned to avoid misalignment. After the identification is completed, the confidence score is calculated by matching with the corresponding textbook corpus. If the confidence score is insufficient, the projection data is used to determine whether it is due to wrinkles, residual distortion of tilt, etc. If so, the correction and identification are re-performed to avoid text cross-line disorder, misjudgment of similar-looking characters, and incorrect matching of pinyin and Chinese characters caused by page tilt.

[0077] Preferably, when collecting the contents of the second-level physical object, two collection triggering methods are supported: one is to use the matching stylus to select the target text paragraph to trigger collection, and the other is to directly scan the whole page and then crop the local image and text area to collect high-definition RGB page images.

[0078] For the new characters with pinyin, a pinyin-Chinese character linkage recognition branch is adopted: first, the pinyin characters above are recognized, and then the corresponding Chinese characters below are matched to correct the recognition errors of similar-looking characters and pinyin tones; for the pure Chinese text sections of the textbook, an OCR model adapted to the font of primary and secondary school textbooks is used to recognize each character and output the complete text with punctuation and line break format.

[0079] It offers two data collection modes: touch-screen stroke selection and full-page scanning and cropping. It is compatible with different learning operations such as short sentence word lookup, whole-text reading, and in-depth reading of difficult parts. It has a low operating threshold and high flexibility.

[0080] Step S300: Based on the recognition results, output basic science information about the corresponding physical object, generating basic recognition knowledge points. The basic science information is obtained by searching existing materials in a third-party knowledge base.

[0081] Step S400: Based on the core theme content, call the corresponding data in the third-party knowledge base to generate core data knowledge points in different dimensions, including background knowledge supplements, core knowledge point memory cards, and graphic and text-based thematic explanation courseware.

[0082] Specifically, by generating keywords for the core topic content, a matching search is initiated with a third-party knowledge base, and the corresponding data is automatically retrieved based on the type of the core topic.

[0083] In a specific instance, the system identifies the physical object "Guanyin Lotus" and automatically retrieves the science encyclopedia sub-database; it also identifies the text "The Rich Xisha Islands" and retrieves the primary and secondary school textbook resource database.

[0084] In step S500, based on basic recognition knowledge points and core data, the panoramic encyclopedia content is loaded synchronously and linked with the desktop projection to generate a VR stereoscopic visualization explanation screen, which is then output to the desktop projection area; learners can learn the knowledge points by manually clicking or through voice interaction.

[0085] Preferably, the panoramic encyclopedia content is loaded simultaneously and linked to a desktop projection to generate a VR stereoscopic visual explanation screen, specifically as follows:

[0086] The system retrieves the 3D data of the corresponding real object from the preset classification library to reconstruct the 3D model, and matches the model texture and structural details with panoramic encyclopedia content. The reconstructed 3D model of the real object is then mapped onto the desktop projection interactive area to generate a VR stereoscopic visualization explanation screen. At the same time, basic reading knowledge points, background knowledge, memory cards, and special topic explanation courseware are superimposed and projected around the VR stereoscopic visualization explanation screen.

[0087] By calling pre-stored 3D data to reconstruct a physical 3D model and matching texture details, a scalable and rotatable VR 3D image is generated by projection. Compared with pure text-based popular science, it intuitively displays the 3D structure of physical objects and reduces the difficulty of understanding abstract knowledge.

[0088] In a specific example, after identifying an orange, a 3D model of the orange is reconstructed, and the projection can be rotated to view the structure of the peel and segments.

[0089] In summary, this application has the following advantages:

[0090] A three-dimensional data classification library covering daily necessities, school supplies, and animals and plants is built in advance, and the names, families, and visual characteristics of the objects are stored in a unified manner. This provides standardized underlying data support for object recognition and three-dimensional display, and solves the shortcomings of traditional reading devices that do not have physical three-dimensional materials and can only recognize single lines of text.

[0091] It integrates multimodal object recognition, hierarchical knowledge point generation, VR stereoscopic projection, and manual / voice interaction into a complete process, forming a closed loop of "object recognition - multi-layered knowledge generation - stereoscopic visualization display - two-way interactive learning", breaking through the problem that traditional devices only output shallow text explanations and have a single dimension of explanation.

[0092] The second aspect of this application provides a three-dimensional knowledge point explanation system, including:

[0093] The classification library creation module is used to collect a certain amount of physical data, including: daily necessities, learning items and three-dimensional data of plants and animals, to create a preset classification library, and to mark the standard name, family or category label and primary visual features of each physical object;

[0094] The image acquisition module is used to place the target to be identified on the desktop projection area, acquire the target image in the desktop projection area, perform multimodal recognition on the target to be identified in the desktop projection area based on a preset classification library, output the specific object identified and the core theme content of the object, and sort out the recognition results.

[0095] The knowledge popularization module is used to output basic popular science introductions of corresponding objects based on the recognition results, and to generate basic recognition knowledge points.

[0096] The extended output module is used to generate core data knowledge points in different dimensions by calling corresponding data from third-party knowledge bases based on core theme content, including background knowledge supplements, core knowledge point memory cards, and graphic and text-based thematic explanation courseware.

[0097] The knowledge explanation module is used to load panoramic encyclopedia content and generate VR stereoscopic visualization explanation screens based on basic recognition knowledge points and core data, and output them to the desktop projection area. Learners can learn knowledge points by manually clicking or through voice interaction.

[0098] Preferably, in the image acquisition module, after the target to be identified is placed on the desktop projection area, the desktop background and the target object foreground are separated by an image semantic segmentation algorithm, the desktop texture, books, and shadow areas are removed, and the object data is extracted.

[0099] Visual feature data are extracted from physical data, including shape and size, surface texture, primary and secondary color schemes, and local key features, to obtain multidimensional visual features;

[0100] The multidimensional visual features are matched with the primary visual features in the preset classification library to obtain the corresponding physical objects. The standard name, family, genus, or category label of the physical objects are then output, and the recognition is completed.

[0101] After recognition is completed, the system synchronously calls up the physical encyclopedia database to load full-dimensional popular science content, and outputs the explanation content in conjunction with the display screen.

[0102] Preferably, in the image acquisition module, multidimensional visual features are matched with first-level visual features in a preset classification library, specifically as follows:

[0103] If a single visual feature has a matching confidence level below the threshold, the projection depth auxiliary information is retrieved for secondary verification. The three-dimensional contour features of the real object are supplemented and re-compared to output the final accurately identified object.

[0104] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A stereoscopic knowledge point explanation method, characterized in that, include: Collect a certain amount of physical data, including: three-dimensional data of daily necessities, learning items and animals and plants, establish a preset classification library, and label each physical object with its standard name, family or category label and primary visual characteristics; The target to be identified is placed on the desktop projection area, and the target image in the desktop projection area is acquired. Based on the preset classification library, the target to be identified in the desktop projection area is recognized in a multimodal manner, and the specific object and the core theme content of the object are output. The recognition results are then sorted out. Based on the recognition results, basic popular science introductions of the corresponding real objects are output, and basic recognition knowledge points are generated. Based on the core theme, corresponding data from third-party knowledge bases are called to generate core data knowledge points in different dimensions, including background knowledge supplements, core knowledge point memory cards and graphic thematic explanation courseware. Based on basic literacy knowledge points and core data, panoramic encyclopedia content is loaded synchronously and linked with desktop projection to generate VR stereoscopic visualization explanation screens, which are then output to the desktop projection area; learners can learn knowledge points by manually clicking or through voice interaction.

2. The stereoscopic knowledge point explanation method of claim 1, wherein, Multimodal recognition is performed on objects within the desktop projection area, specifically as follows: After placing the target to be identified on the desktop projection area, the desktop background and the target object foreground are separated by the image semantic segmentation algorithm. Desktop texture, books, and shadow areas are removed, and object data is extracted. Visual feature data are extracted from physical data, including shape and size, surface texture, primary and secondary color schemes, and local key features, to obtain multidimensional visual features; The multidimensional visual features are matched with the primary visual features in the preset classification library to obtain the corresponding physical objects. The standard name, family, genus, or category label of the physical objects are then output, and the recognition is completed. After recognition is completed, the system synchronously calls up the physical encyclopedia database to load full-dimensional popular science content, and outputs the explanation content in conjunction with the display screen.

3. The stereoscopic knowledge point explanation method of claim 2, wherein, The multidimensional visual features are matched with the first-level visual features in the preset classification library, specifically as follows: If a single visual feature has a matching confidence level below the threshold, the projection depth auxiliary information is retrieved for secondary verification. The three-dimensional contour features of the real object are supplemented and re-compared to output the final accurately identified object.

4. The three-dimensional knowledge point explanation method as described in claim 2, characterized in that, The physical objects are divided into primary objects with no readable content on their surfaces and secondary objects with readable content on their surfaces. The secondary objects include textbooks, books, and picture books. The surfaces of the secondary objects are further scanned using a scanning device, and text and image recognition is performed to obtain secondary visual features, which are then used as the core theme content.

5. The three-dimensional knowledge point explanation method as described in claim 4, characterized in that, The secondary physical surface is further scanned using a scanning device. The specific method is as follows: After the scanning device is started, the effective interactive recognition area is delineated by desktop projection. After placing an open secondary physical object with readable content, the posture of the book page is corrected by adjusting the projection frame of the scanning device, and the open content of the secondary physical object is collected. The semantic segmentation model is used to process the contents of the second-level physical object, including: dividing the vocabulary list, text, after-class exercises, short writing exercises, and illustration areas, and removing illustrations and decorative patterns; at the same time, restoring the line spacing and paragraph segmentation information of the original layout, distinguishing the hierarchical structure of the pinyin column, Chinese character column, and question and answer column, and obtaining the text recognition content. The system calls upon third-party knowledge bases, including: vocabulary databases and text corpora from primary and secondary school textbooks, and performs corresponding verification between the text recognition content and the third-party knowledge bases to match the original texts of standard textbooks or extracurricular reading materials for the corresponding grade level; at the same time, it provides labels for the text recognition content.

6. The three-dimensional knowledge point explanation method as described in claim 5, characterized in that, When processing the unfolded contents of the secondary physical object using a semantic segmentation model: The text lines are skewed and split into lines, breaking the entire page of text into a sequence of single-line text. For bulleted sentences such as after-class Q&A and short writing exercises, retain the original bullet points and line breaks, and restore the format of the questions and guiding example sentences; When setting a single-match confidence level, if the confidence level of the text recognition content is insufficient when verifying its correspondence with a third-party knowledge base, it is determined to be non-text, and the projection depth information is combined to distinguish between book pages and the lower desktop.

7. The three-dimensional knowledge point explanation method as described in claim 5, characterized in that, When collecting the contents of the second-level physical object, two collection triggering methods are supported: one is to use the matching stylus to select the target text paragraph to trigger collection, and the other is to directly scan the whole page and then crop the local image and text area to collect high-definition RGB page images. For the new characters with pinyin, a pinyin-Chinese character linkage recognition branch is adopted: first, the pinyin characters above are recognized, and then the corresponding Chinese characters below are matched to correct the recognition errors of similar-looking characters and pinyin tones; for the pure Chinese text sections of the textbook, an OCR model adapted to the font of primary and secondary school textbooks is used to recognize each character and output the complete text with punctuation and line break format.

8. The three-dimensional knowledge point explanation method as described in claim 1, characterized in that, The panoramic encyclopedia content is loaded synchronously and linked to a desktop projection to generate a VR stereoscopic visual explanation screen, specifically: The system retrieves the 3D data of the corresponding real object from the preset classification library to reconstruct the 3D model, and matches the model texture and structural details with panoramic encyclopedia content. The reconstructed 3D model of the real object is then mapped onto the desktop projection interactive area to generate a VR stereoscopic visualization explanation screen. At the same time, basic reading knowledge points, background knowledge, memory cards, and special topic explanation courseware are superimposed and projected around the VR stereoscopic visualization explanation screen.

9. A three-dimensional knowledge point explanation system, employing the method described in any one of claims 1-8, characterized in that, include: The classification library creation module is used to collect a certain amount of physical data, including: daily necessities, learning items and three-dimensional data of plants and animals, to create a preset classification library, and to mark the standard name, family or category label and primary visual features of each physical object; The image acquisition module is used to place the target to be identified on the desktop projection area, acquire the target image in the desktop projection area, perform multimodal recognition on the target to be identified in the desktop projection area based on a preset classification library, output the specific object identified and the core theme content of the object, and sort out the recognition results. The knowledge popularization module is used to output basic popular science introductions of corresponding objects based on the recognition results, and to generate basic recognition knowledge points. The extended output module is used to generate core data knowledge points in different dimensions by calling corresponding data from third-party knowledge bases based on core theme content, including background knowledge supplements, core knowledge point memory cards, and graphic and text-based thematic explanation courseware. The knowledge explanation module is used to load panoramic encyclopedia content and generate VR stereoscopic visualization explanation screens based on basic recognition knowledge points and core data, and output them to the desktop projection area. Learners can learn knowledge points by manually clicking or through voice interaction.

10. The three-dimensional knowledge point explanation system as described in claim 9, characterized in that, In the image acquisition module, after the target to be identified is placed on the desktop projection area, the desktop background and the target object foreground are separated by the image semantic segmentation algorithm, the desktop texture, books, and shadow areas are removed, and the object data is extracted. Visual feature data are extracted from physical data, including shape and size, surface texture, primary and secondary color schemes, and local key features, to obtain multidimensional visual features; The multidimensional visual features are matched with the primary visual features in the preset classification library to obtain the corresponding physical objects. The standard name, family, genus, or category label of the physical objects are then output, and the recognition is completed. After recognition is completed, the system synchronously calls up the physical encyclopedia database to load full-dimensional popular science content, and outputs the explanation content in conjunction with the display screen.

11. The three-dimensional knowledge point explanation system as described in claim 10, characterized in that, In the image acquisition module, multi-dimensional visual features are matched with first-level visual features in a preset classification library, specifically as follows: If a single visual feature has a matching confidence level below the threshold, the projection depth auxiliary information is retrieved for secondary verification. The three-dimensional contour features of the real object are supplemented and re-compared to output the final accurately identified object.

Images